Most mistranscriptions come from four places: microphone gain that's clipping or too quiet, room acoustics bouncing your voice around, a Whisper model too small for the job, or vocabulary — names, jargon, product terms — the model has never seen. Fix those four in order and the error rate usually drops enough to make dictation usable for real work, not just short commands.
It's rarely the transcription engine itself that's broken. Whisper and its derivatives are trained on a huge corpus of general speech, so they're good at "the" and "meeting" and bad at "Kubernetes" or your colleague's surname. Treating every misheard word as a software bug means you keep fighting symptoms instead of the actual input problem.
Speech recognition, on-device or cloud, is a garbage-in-garbage-out system. A model can only work with the audio it's handed — if that audio is clipped, muffled by room echo, or picking up your fan more than your voice, no amount of parameter count fixes it downstream.
This matters more for on-device tools than cloud ones, and it's worth being honest about it. Cloud speech APIs run enormous models with server-grade compute behind them; a local Whisper model on your laptop is working with a fraction of that capacity. That's the tradeoff for keeping your voice off someone else's server — you get privacy and offline reliability, but you also need to feed the model cleaner audio to hit the same accuracy. The good news is that cleaning up your input is a five-minute fix, not a hardware upgrade.
1. Microphone gain. Open your system's sound settings (or pavucontrol on PulseAudio/PipeWire systems) and watch the input level while you talk normally. If it's pinned in the red, you're clipping — the waveform is getting cut off and the model is guessing at missing information. If it barely moves, the model is transcribing mostly noise floor. Aim for peaks in the upper-middle of the meter, not maxed out.
2. Microphone choice and position. Laptop built-in mics are omnidirectional and pick up keyboard clatter, fan noise, and room reflections along with your voice. A cheap USB or headset mic with a cardioid pickup pattern, positioned 10-15cm from your mouth, will outperform a built-in mic on a $2,000 laptop every time. This is the single highest-leverage fix most people skip.
3. Room acoustics. Hard, parallel surfaces — bare walls, glass desks, empty rooms — create reflections that smear consonants together, which is exactly what voice activity detection (VAD) and the transcription model both struggle with. You don't need acoustic foam; a rug, curtains, or even a bookshelf behind you cuts reflections enough to matter. Test by clapping once in your room — a long, ringy decay means the room is working against you.
4. Model size. Whisper ships in tiers — tiny, base, small, medium, large — and there's a real accuracy-for-speed tradeoff. Tiny and base are fast enough to feel instant but miss more on accents, fast speech, and jargon. Small and medium are noticeably more accurate but take longer to transcribe and use more CPU/GPU. If you're on a machine with a decent GPU or even a modern CPU, bumping up a tier is often the single biggest accuracy gain available to you, at the cost of a slightly longer pause after you stop talking.
No amount of model size fixes a word the model has never encountered in training. Product names, coworkers' names, internal jargon, and technical terms outside common usage will get mangled by even the largest Whisper model, because the training data simply didn't contain them at the frequency needed to learn them reliably.
The fix is a word-replacement dictionary — a simple find-and-replace layer that runs after transcription. You type "kubernetes" gets misheard as "cooper netties" once, add a rule mapping it back, and it's fixed permanently for every future dictation session. This is mechanically simple — rule-based string substitution, not another AI call — which is exactly why it's reliable and works offline. Voxtty runs this as part of its post-processing: press Alt+D, speak, faster-whisper transcribes on-device, and a rule-based cleanup pass (plus optional custom replacements) fixes recurring misses before the text lands in whatever app has focus. If dictation accuracy is the thing standing between you and using voice input daily, try Voxtty free and see whether the gain/acoustics/model-size combination gets you there.
Be honest with yourself about a few hard limits. Strong regional accents and non-native speech patterns are still where local Whisper models lag furthest behind top-tier cloud STT — the training data skews toward standard American and British English, and a small local model won't close that gap the way a much larger cloud model might. Background noise from open-plan offices, other people talking, or reverberant rooms will degrade any model, local or cloud.
Rapid speech, mumbling, and mid-sentence self-correction ("send it to— actually cc Dana too") also trip up VAD-based systems, since the silence detection that decides where one utterance ends and the next begins wasn't designed to parse interruptions. And a wake-word style "always listening" mode is inherently harder to get right than a manual hotkey — it has to distinguish your trigger phrase from every other sound in the room, which is why push-to-talk hotkeys like Alt+D remain more reliable than always-on listening for accuracy-sensitive work.
Before touching any settings, do the clap test in your usual dictation spot, then check your mic's input level while talking at normal volume. If either looks off — long echo decay or a meter pinned in the red — fix that first. It's a five-minute change that fixes more mistranscriptions than any model upgrade will.
Local-first voice dictation for Linux. Press Alt+D, speak, and your words land in whatever app has focus — nothing leaves your machine.
Try Voxtty free →