Voice activity detection (VAD) is the process a dictation tool uses to figure out, from the raw audio signal, when you've started talking and when you've stopped — without relying on a fixed clock. Instead of waiting for "3 seconds of silence" to decide you're done, VAD models the audio itself: energy levels, spectral shape, sometimes a small neural classifier trained to distinguish speech from breathing, keyboard clatter, or a fridge compressor. That distinction is why some dictation tools feel like they're keeping up with you and others feel like they're fighting you.
A fixed-timeout approach is simple to build: start recording, wait N seconds after audio drops below some volume, then stop and transcribe. The problem is that human speech doesn't move at a constant pace. You pause to think mid-sentence, especially when dictating something technical — a variable name, a clause you're still constructing in your head.
Set the timeout short and the tool cuts you off mid-thought, splitting one sentence into two garbled fragments. Set it long and you get dead air at the end of every utterance — a beat of silence before the text appears, which breaks the flow of typing at speed. Neither failure mode is rare; they're the default behaviour of any timer-based cutoff, because a timer can't tell the difference between "I'm thinking" and "I'm finished."
Real VAD systems work on short audio frames — typically 10 to 30 milliseconds — and classify each one as speech or non-speech. Simple implementations use signal energy and zero-crossing rate: speech has a distinctive energy pattern and a spectral signature different from silence or steady background noise. More capable ones use a small trained model (this is the approach in tools like Silero VAD, which faster-whisper's pipeline commonly pairs with) that's been trained on labelled speech and non-speech audio to make that call frame by frame.
The output isn't just a binary flag — it's a stream of decisions that gets smoothed with a bit of hysteresis. Most implementations require several consecutive speech frames before declaring "speech started," which avoids triggering on a single cough or a chair creak. On the tail end, they apply a short "hangover" window — holding the speech state open for a few hundred milliseconds after audio drops — so a natural mid-sentence pause doesn't get read as the end of your utterance.
This is the mechanism Voxtty relies on: press Alt+D, start talking, and VAD tracks your actual speech boundaries rather than counting down a timer. Recording stops when you stop talking, not when an arbitrary clock expires, and the audio is transcribed on-device with faster-whisper before the text lands wherever your cursor is. If you want to see how that feels in practice, try Voxtty free.
VAD sensitivity is a tuning knob, and every position on it trades one failure mode for another. Set it aggressive — quick to detect speech end — and you get fast, snappy transcription, but you'll clip the tail off sentences that end quietly, and a sigh or a sniff can look enough like the start of speech to trigger a false start. Set it conservative and you get fewer false starts and cleaner sentence boundaries, but longer hangover time means more dead air waiting for silence to confirm you're actually done.
There's no universally correct setting because it depends on how you talk. Someone who dictates in short, clipped commands wants a tighter hangover window. Someone composing longer, more considered prose — with natural thinking pauses — wants more slack before the tool decides they've finished.
VAD isn't perfect, and it's worth being upfront about where it breaks down. Noisy environments are the main one: a running fan, traffic through an open window, or a mechanical keyboard's clatter can sit close enough to speech-like energy patterns that energy-based VAD gets confused, either triggering falsely or failing to detect quieter speech underneath the noise. Neural VAD models handle this better than pure energy thresholds, but no model is immune to a genuinely loud room.
Long, deliberate pauses mid-thought — the kind where you stop for several seconds to figure out the next sentence — are also a genuine edge case, because any hangover window has to end somewhere, and a wait long enough to cover every possible pause would make every dictation session feel sluggish. And VAD only decides when you're speaking, not what you said — accented speech, jargon, or crosstalk with someone else in the room are transcription accuracy problems, not VAD problems, and Whisper models have known limits there too.
If you already dictate on Linux, or you're curious whether it's worth the switch from typing, notice the next time a tool cuts you off mid-sentence or leaves a beat of silence before your words appear — that's VAD tuning, not bad luck. Try speaking one paragraph at a natural pace, including a mid-sentence pause, and see whether the tool keeps up or breaks it into fragments. That single test tells you more about a dictation tool's VAD than any spec sheet.
Local-first voice dictation for Linux. Press Alt+D, speak, and your words land in whatever app has focus — nothing leaves your machine.
Try Voxtty free →