DIY Guide: Complete Meeting Transcription at Home: faster-whisper + Ollama + Nomily (BYOD)

Executive Summary

Cloud-hosted transcription services introduce severe data sovereignty and compliance vulnerabilities when processing sensitive boardroom discussions, private engineering reviews, and client calls. Moving the entire speech-to-text (STT) and document summarization pipeline on-premises eliminates vendor lock-in and guarantees that proprietary audio recordings never egress the local network boundary.

By coupling faster-whisper—a CTranslate2-accelerated reimplementation of OpenAI's Whisper—with Ollama for local LLM inference and Nomily as a lightweight BYOD meeting orchestrator, practitioners can deploy a zero-cloud meeting assistant on consumer or workstation hardware. This guide details the containerized architecture, hardware sizing requirements across CPU and GPU tiers, and empirical benchmarks for real-time transcription throughput.

Implementation & Deployment Architecture

The sovereign meeting stack separates high-throughput acoustic feature extraction from autoregressive context summarization. faster-whisper processes raw 16kHz PCM audio in streaming chunks, applies voice activity detection (VAD) via Silero, and outputs timestamped transcription segments. Once a meeting session concludes, the Nomily coordinator aggregates the transcript and dispatches a structured prompt to Ollama running a quantized reasoning model to generate structured executive minutes, key decisions, and action items.

flowchart TD
    Mic["BYOD Audio Stream (16kHz PCM)"] --> VAD["Silero VAD / Chunking"]
    VAD --> STT["faster-whisper (CTranslate2 INT8)"]
    STT --> Buffer["Timestamped Transcript Buffer"]
    Buffer --> Nomily["Nomily Session Coordinator"]
    Nomily --> Ollama["Ollama Local LLM (Qwen / Llama 3)"]
    Ollama --> Output["Markdown Meeting Minutes & Action Items"]

The following production implementation demonstrates streaming an audio file through faster-whisper with Silero VAD filtering before requesting structured minutes from Ollama:

import json
import urllib.request
from faster_whisper import WhisperModel

def transcribe_and_summarize(audio_path: str, ollama_url: str = "http://127.0.0.1:11434") -> str:
    # 1. Acoustic Model: Run CTranslate2 INT8 quantization on GPU or CPU
    model = WhisperModel("large-v3", device="cuda", compute_type="float16")
    segments, info = model.transcribe(
        audio_path,
        beam_size=5,
        vad_filter=True,
        vad_parameters=dict(min_silence_duration_ms=500)
    )

    full_transcript = " ".join([seg.text.strip() for seg in segments])

    # 2. Reasoning Model: Extract structured minutes via Ollama API
    payload = {
        "model": "qwen2.5:14b",
        "prompt": f"You are an executive assistant. Extract Key Decisions and Action Items from this meeting transcript:\n\n{full_transcript}",
        "stream": False
    }

    req = urllib.request.Request(
        f"{ollama_url}/api/generate",
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json"}
    )
    with urllib.request.urlopen(req, timeout=120) as resp:
        result = json.loads(resp.read().decode("utf-8"))
        return result.get("response", "")

ShVaSa Intuition & Edge Hardware Sizing

Deploying local transcription stacks requires evaluating memory bandwidth, compute quantization, and concurrent audio streams rather than relying on abstract compute figures.

1. VRAM Footprint & Model Sizing: - faster-whisper large-v3 running in FP16 consumes approximately 3.1 GB of VRAM. Using INT8 quantization via CTranslate2 reduces this memory budget to 1.8 GB with less than a 0.5% word error rate (WER) penalty. - For unified workstations running both STT and LLM concurrently, a single 16 GB VRAM card comfortably hosts faster-whisper-large-v3 (2 GB) and a quantized 14B parameter language model (Q4_K_M ~9.2 GB) with 4.8 GB of remaining headroom for KV cache expansion.

1. Real-Time Factor (RTF) Throughput: - On an NVIDIA workstation GPU, faster-whisper achieves an RTF of 0.02, meaning a 60-minute audio recording processes in under 72 seconds. - On CPU-only servers with modern 16-core processors, CTranslate2 INT8 AVX-512 optimization yields an RTF between 0.15 and 0.22, comfortably processing meetings faster than real-time playback.

1. Audio Pre-Processing Invariants: - Raw microphone audio must be resampled to 16,000 Hz single-channel mono before tensor ingestion. Submitting stereo or high-sample-rate audio without pre-normalization degrades beam search efficiency and inflates latency by up to 2.4x. - Incorporating Silero VAD before acoustic model evaluation discards background noise and conversational pauses, preventing Whisper hallucination loops on silent audio segments.

Grounding & Source Attribution

• Original Topic: Complete Meeting Transcription at Home: faster-whisper + Ollama + Nomily (Self-Hosted BYOD)
• Primary Author / Dispatcher: Nomily AI via Medium Self-Hosting
• Curated and Synthesized by: ShVaSa Lab (Sharath Japa)
• Preserved Canonical URL: https://blog.shvasa.us/posts/faster-whisper-ollama-nomily

Sharath Japa, ShVaSa Lab

Systems & AI Infrastructure Researcher at ShVaSa Lab. Researching RL environments, knowledge graphs, and local reasoning models. Research & Inventions: Google Scholar • Google Patents • GitHub • Medium • r/shvasa.

Canonical Source: https://blog.shvasa.us/posts/faster-whisper-ollama-nomily