Dictation Without Filler Words: Local Whisper and Claude for Cleanup

Live speech is never clean: it's full of filler words, cut-off phrases, self-corrections. A developer from the HSE Graduate School of Business, who writes books and dictates a lot, built a system that turns "So, like, I, um, wanted to say that, well, tomorrow's call is moving, uh, to three, no, to four" into the clean sentence "I wanted to say that tomorrow's call is moving to four" — in 2 to 3.5 seconds.

Local Transcription and Cloud-Based Cleanup

The dictation runs on Handy, an open-source app built on Tauri (MIT license). Speech is transcribed by Whisper Large v3 Turbo directly on the computer, so the audio never leaves the machine. On an RTX 4080 Super GPU, 8 seconds of speech are transcribed in 0.17 seconds, and 42 seconds take 0.45 seconds. For Russian, the system uses the klava-nevinovata build, which comes with ready-made Whisper prompts and protection against looping on silence.

Whisper alone isn't enough: a simple find-and-replace list doesn't work — the system needs to understand meaning. The transcribed text is sent to Claude Sonnet 5.5, which removes filler words, fixes self-corrections, and preserves the conversational tone. Cleanup takes 1.7 to 3 seconds depending on the length of the text.

The Middleware Layer and Context-Aware Dictionary

The middleware tool, Chistovik — a single Python file using only the standard library — sits between Handy and the API. This layer has several advantages: the prompt and dictionary live in text files that are re-read before every dictation (you can edit a rule, dictate again, and it just works — no restart needed); it keeps one warm connection instead of spinning up a new HTTP client for every request; the API key lives in an environment variable rather than the app's settings; and a 10-second timeout prevents hangs.

The dictionary understands context. Whisper mishears rare words — it hears "AI-ish" as "egg-ish" in Russian. A blind replacement would be harmful, since the word might be used literally. Instead, the model is given a condition: "AI-ish, AI-ish-es, AI-ishly ← mishearing, egg-ish (replace 'egg-ish' only when the text is not about eggs or food)." Claude understands these instructions equally well in Russian or English. Tested case: in "egg salad recipe," the word stays as is (the text is about food); in "the text came out kind of egg-ish," it's corrected to "AI-ish" (the text is about AI).

Integration Bugs and Their Fixes

During development, three classes of problems emerged. The model sometimes treated dictated text as a request to act on — fixed by wrapping the text in a <dictation> tag and adding a clear rule in the prompt that requests should never be executed. The model sometimes swapped casual words for formal ones — fixed with examples baked into the prompt itself: "cool" stays "cool," "whatever" stays "whatever." Post-processing errors were invisible, since Handy would silently insert the raw, unprocessed text — the middleware now logs every dictation. On Windows, localhost can resolve to IPv6, so the middleware listens on both 127.0.0.1 and ::1.

Performance and Takeaways

The full pipeline, from keypress to inserted text:

• 8 seconds of speech: 0.17s for transcription, 1.7s for cleanup, roughly 2 seconds total
• 42 seconds of speech: 0.45s for transcription, 3s for cleanup, roughly 3.5 seconds total

Each API request runs about 1,100 tokens, almost all of it rules and dictionary entries. The code is open source (MIT) at github.com/Tataut/chistovik and installs with a single PowerShell script on Windows. The middleware itself is cross-platform, and Handy is also available for macOS and Linux. Any OpenAI-compatible API works: OpenAI, OpenRouter, or local models via Ollama or LM Studio (with local models, nothing leaves the computer). The system has been tested with Claude Sonnet 5.5. It's most useful in messaging apps and Telegram.