Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)

A 35-second voice message becomes text in 9 seconds, no GPU required

Customers send voice messages like "Is that set still available, and how much is delivery to Samara?", and a manager on a noisy shop floor can't listen to them. So the sales CRM shows a text transcript under the player, generated by Whisper small on a cloud server in Russia with 2 CPU cores and no GPU. A 35-second voice message turns into text in about 9 seconds, and the audio never leaves to foreign clouds. For a small message volume, this works without paid APIs.

AI-processed from Хабр: Машинное обучение; edited by Hamidun News
A 35-second voice message becomes text in 9 seconds, no GPU required
Source: Хабр: Машинное обучение. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceХабр: Машинное обучение ↗
Editor-in-Chief
Key metric #1int8 quantization and greedy decoding delivered the main speedup on CPU
Key metric #2PyAV failed on some ogg, m4a and mp3 files, so ffmpeg handles audio decoding
Key metric #3A cron job runs once a minute, picks up to 5 voice messages, and gives each file 3 attempts

Customers of online stores love voice messages: "Hello, is this set still available, and how much is delivery to Samara?" A manager has to listen to such a message, which is inconvenient on a noisy shop floor or during a call. The team behind the sales CRM decided to show a ready-made text under the player. All transcription runs on an ordinary cloud server in Russia with two CPU cores and no GPU: a 35-second voice message turns into text in about 9 seconds, which is nearly 4 times faster than the recording itself lasts. There's no need to pay cloud recognition services for minutes of audio, and the customer's audio never leaves the company's servers.

Why not use an existing solution

The first idea was to use the messenger's own transcription. VKontakte has one: the API response for an attachment includes a field with the text. But for community messages, that field stayed empty even several days later. For MAX, Telegram and WhatsApp, which are connected through an intermediary service, there is no transcription at all.

Cloud recognition services were ruled out for a different reason. A customer's voice message is personal data, and the team did not want to send it to foreign servers abroad. So everything has to run on the company's own servers in Russia, and files are not stored anywhere.

What's under the hood

For recognition, the team chose faster-whisper, an implementation of the Whisper model on CTranslate2 that is noticeably faster than the original on an ordinary CPU. They used the small version of the model. On conversational Russian it makes errors on individual words, but it conveys the meaning reliably, and a manager doesn't need more than that.

Recognition was moved to a separate machine so it wouldn't take processor time away from the CRM itself. Four settings provided the speed:

  • weight quantization (int8, meaning the model's numbers are stored in compressed form), the main speed gain on CPU;
  • greedy decoding instead of beam search: on short phrases, quality barely changes;
  • a voice activity detector trims silence and pauses, of which voice messages contain plenty;
  • the language is set explicitly, without auto-detection: this is faster and yields fewer errors on short recordings.

Pitfalls with audio decoding

faster-whisper can open files itself through the PyAV library, but this broke on voice messages from messengers. On some files PyAV crashed with a metadata read error, and the formats vary: ogg, m4a, mp3.

The fix was simple: hand decoding to the ffmpeg program and pass the model an already prepared audio array: mono, 16 kHz, float32 values from −1 to 1. This way, the source file format no longer matters.

The script is built on the "stdin → stdout" principle: audio comes in, text goes out. It knows nothing about the CRM or the messengers, and the file never touches the disk. The main server calls it with a single command via curl and ssh.

How it's built into the CRM

On the main server, a cron job runs once a minute. It is wrapped in flock: if the previous run is still going, a new one does not start, so the queue doesn't pile up. A timeout of 600 seconds guards against a hung ssh or ffmpeg. Each pass processes no more than five voice messages, so a burst of messages doesn't overwhelm the recognition server.

The job finds incoming voice messages from the last 2 days that don't yet have text, downloads the audio one by one from the messenger's link, sends it for recognition, and writes the result into the message attachment. The CRM interface shows the text under the player, and in the dialog list, instead of "Voice message," the preview reads "🎤 Hello, is this set still…".

Each message has an attempt counter: if a file is corrupted or a link has expired, the message is no longer touched after three failures. The 2-day window keeps the job from scanning the entire message history on every run. The text appears about a minute after the voice message arrives: the wait until the next run plus a few seconds for recognition.

Limitations and conclusion

The honest downside of this scheme: the model loads on every call. For a small volume of voice messages this is acceptable, but under heavy load the script should be turned into a persistently running service with a queue. The small model's quality on conversational speech is also not perfect: some words get distorted.

The recipe for those who want to replicate it: use faster-whisper with int8 on CPU, decode audio through ffmpeg rather than PyAV, and don't store files, since transcription doesn't need them. For small and medium-sized businesses, this is an example of how to handle a task involving personal data without a GPU and without paying for minutes of audio.

Story

Key Questions Answered

Why not use an existing solution?

The first idea was to use the messenger's own transcription. VKontakte has one: the API response for an attachment includes a field with the text. But for community messages, that field stayed empty even several days later. For MAX, Telegram and WhatsApp, which are connected through an intermediary service, there is no transcription at all. Cloud recognition services were ruled out for a different reason. A customer's voice message is personal data, and the team did not want to send it to foreign servers abroad.

What's under the hood?

For recognition, the team chose faster-whisper, an implementation of the Whisper model on CTranslate2 that is noticeably faster than the original on an ordinary CPU. They used the small version of the model. On conversational Russian it makes errors on individual words, but it conveys the meaning reliably, and a manager doesn't need more than that. Recognition was moved to a separate machine so it wouldn't take processor time away from the CRM itself.

What are the key details on Pitfalls with audio decoding?

faster-whisper can open files itself through the PyAV library, but this broke on voice messages from messengers. On some files PyAV crashed with a metadata read error, and the formats vary: ogg, m4a, mp3. The fix was simple: hand decoding to the ffmpeg program and pass the model an already prepared audio array: mono, 16 kHz, float32 values from −1 to 1. This way, the source file format no longer matters. The script is built on the "stdin → stdout" principle: audio comes in, text goes out.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…