ElevenLabs Scribe v3: 27% Fewer Transcription Errors and Live Speaker Labeling

Every voice product starts with a transcript. If "fifteen" turns into "fifty" there, the error will travel on to the summary, the CRM and the analytics model, and fixing it at that point will cost many times more. ElevenLabs has released Scribe v3 and Scribe v3 Live, two speech recognition models built for real-world mess: call participants interrupt each other, a customer goes quiet in the middle of a phone number, and both parties are speaking in a language that isn't their native one.

What has changed

Scribe v3 is designed for recorded audio. According to the company, across 102 languages it makes 27% fewer word errors than Scribe v2 and has improved in every language tested. German improved by 42%, Spanish by 35%, English by 23%, Swahili by 34%, Tamil by 29% and Thai by 28%. Diarization (labeling "who said what") became 12% more accurate overall and 17% more accurate on recordings with three or more participants. Those are exactly the recordings where most models break down.

Scribe v3 Live is the low-latency streaming variant. It makes 43% fewer errors than Scribe v2 Realtime by semantic WER (which counts as an error only what changes the meaning). There are two main new features. The first is real-time diarization: 3.1% errors on the English portion of the company's own dataset, versus 5.9–7.3% for other streaming models. The second is improved end-of-turn detection.

A classic voice agent waits for a pause of a set length. With a short timer, it interrupts a person who is pausing to think in the middle of a card number. With a long one, the agent seems sluggish. Scribe v3 Live takes into account not only silence but also the context of the conversation, so a pause inside a number isn't treated as the end of a sentence. The same logic was already working in ElevenAgents, and now it can be hooked up through the API (the commitStrategy parameter with the value turn_prediction).

There is also more control over the output. Keyterm prompting accepts up to 1,000 terms that are sure to come up: brand names, abbreviations, jargon. Numbers can be returned as digits or as words. Formatting rules are set in plain language, for example "write all dates as YYYY-MM-DD." Live gains a multilingual mode for conversations where the language changes mid-sentence, such as Hindi mixed with English. It also has a keyterm library: it holds thousands of terms that might come up, and the model consults it only when it hears a likely match. This is handy for a product catalog or a list of medicines.

What a few percent is worth

Percentages in release notes are hard to feel, so ElevenLabs translates them into minutes. An hour-long meeting at 150 words per minute is about 9,000 words. The difference between 4% and 2% WER is 180 wrong words, or three for every minute of speech. Diarization is measured in time: each point of error means 36 seconds of speech attributed to the wrong person.

The price of an error depends on where it lands. In a podcast transcript, a wrong word is a typo. In an earnings report, "fifteen" heard as "fifty" ends up in an analyst's model. Errors also cluster in names, numbers, abbreviations and jargon — precisely where a summarizer or an agent needs maximum accuracy.

A real-world example: the Dutch energy company Eneco (more than 2 million customers) tested Scribe v3 Live in its voice agent on real calls. Dutch postal codes were recognized correctly on the first attempt 40% more often, and house numbers 15% more often.

Caveats and what to do

All the figures are ElevenLabs' own measurements. Multilingual accuracy was calculated on an internal version of FLEURS, which is short, clean read speech, far from a noisy call center. For diarization, the company built its own dataset of real meetings, agent calls and voice notes in 16 languages and promises to open it "soon." Until it is released, the results cannot be independently verified. In addition, errors were averaged equally across languages, while many publications merge all clips into a single metric, so a direct comparison with other people's figures is not valid. Real-time diarization also cannot be compared with batch diarization: a different counting methodology was used there.

The practical takeaway is simple. If you are on Scribe v2, switching takes one line: the model scribe_v3 or scribe_v3_live. But don't take the percentages on faith. Take 50–100 of your own real recordings, especially ones with numbers, names and language switching, and compare them using your own metric. Teams with voice agents should work out in advance how often the agent interrupts people and mixes up digits: this is where the gain is most noticeable.

There are extras for enterprise. By default, data is processed in the US, but isolated environments are available in the EU, India and Singapore, along with a Zero Retention mode that deletes audio and text immediately after the request, SOC 2, ISO 27001 and PCI DSS Level 1 certifications, HIPAA with a BAA, and private deployment. For healthcare, the company is also releasing a Live variant optimized for medical conversations.