ElevenLabs Teaches Voice Agents to Hear Drug Names More Accurately: 17% Fewer Errors
A voice agent writes into the patient record whatever it hears, and drug names are easy to confuse: one model mistook sitagliptin, a diabetes drug, for citalopram, an antidepressant. ElevenLabs has released Scribe v3 Live Medical, speech recognition built for live clinic calls. On the PriMock57 test (57 simulated patient visits), 12.8% of words came out wrong, which is 17% fewer errors than the closest medical model. For teams building medical voice agents, it is a reason to test the model on their own calls.
AI-processed from ElevenLabs Blog; edited by Hamidun News
A voice agent in a clinic acts on what it hears, so a single misrecognized phrase becomes a wrong entry in the record. If a patient says they take sitagliptin for diabetes, the agent may add that drug to the medication list the doctor sees. But in ElevenLabs' tests, one of the speech models heard "citalopram" instead, an antidepressant and a completely different drug.
To address such errors, the company has released Scribe v3 Live Medical, a medical version of its real-time speech recognition model. On the PriMock57 test (57 simulated visits to a general practitioner), it has a word error rate of 12.8%, or roughly 13 wrong words out of 100.
That is 17% fewer errors than the closest medical model in ElevenLabs' own comparison. For a voice-agent developer, the takeaway is simple: drug names, dosages and policy numbers break most often at the recognition step, and this model breaks on them less.
What the model can do
Scribe v3 Live Medical is built on the new Scribe v3 Live model and is already available in the ElevenAgents platform: open Automatic Speech Recognition in the settings and pick it from the list. Previously, ElevenLabs offered medical transcription only for recordings, such as dictations and visits. Now it works on live calls: appointment booking, information lines, phone insurance verification, and follow-ups on claims. Such calls cannot wait for the recording to end, and drug names and dosages rarely appear in everyday speech, which is why general-purpose models make the most mistakes on them.
The first tool is a custom dictionary of up to 10 000 terms. A regional clinic network can upload the names of all its doctors and branch locations, and a pharmaceutical company can upload its entire portfolio under both brand and international nonproprietary names. The model consults the list only when the caller appears to be saying one of those terms. According to ElevenLabs, the dictionary cuts errors on medical terms by up to 48%.
The second is careful handling of numbers. People read out a policy number or date of birth in chunks, pausing while they look for the next group on their card. A typical agent treats the pause as the end of the turn and interrupts, and the person has to start over. Here, the model recognizes that the number is not finished and waits for the rest.
The third is switching languages mid-conversation. A patient may move to their native language to describe a symptom better. The model supports more than 90 languages and does not lose such fragments.
What the tests showed
The company compared the model against streaming models from leading providers, both medical and general-purpose, on the same recordings. It chose PriMock57 because it contains everything that makes live calls hard: unscripted conversation, phone audio with background noise, and dense clinical detail.
On Fareez OSCE (272 recordings of training patient interviews), the model has the lowest "semantic" error rate among medical models, at 0.6%. This metric counts only errors that change the meaning: "20 mg" and "twenty milligrams" count as a match, which is why its figures are lower than those of the ordinary word error rate. On the Eka Medical ASR set (more than 3 600 English recordings of doctors and students, with brand-name drugs), it makes 4.7% fewer errors than regular Scribe v3 Live: 11% versus 11.54% on the semantic metric.
For multilingual clinics, there is the MED-Term test from Corti, which uses rare terms: the semantic error rate is 2.24% in English, 2.81% in German and 3.24% in French. On ordinary speech (Common Voice and FLEURS), the model performs about the same as the base version, so it can be enabled for all calls, including rescheduling a visit and questions about opening hours. Latency stayed at about 150 ms, so the clinical specialization does not add a noticeable pause.
ElevenLabs translates this into everyday numbers. A 10-minute call contains about 1 500 words, and the model recognizes roughly 40 more words correctly than the closest medical competitor. At 1 000 calls a day, that is about 40 000 words. Not all words are equal: a dropped "um" is harmless, while a mixed-up drug ends up in a prescription request, and a wrong digit in a policy number delays the insurance check.
Privacy and caveats
The model meets HIPAA requirements for enterprise customers with a signed BAA and Zero Retention mode enabled (data is not stored). ElevenAgents holds SOC 2 Type II, ISO 27001 and PCI DSS Level 1 certifications. Patient data is stored in the US by default, and isolated environments in the EU, India or Singapore can be used if needed.
A caveat: all comparisons were run by ElevenLabs itself, on providers it selected and in the best configuration for each. The competitors are not named, and the list of models is incomplete. Among its customers, the company names Pfizer, which notes accuracy on brand and international names, and Vintage Labs, where, according to the founder, nurses trust the call summary without re-listening to the recording.
The practical takeaway for anyone building medical voice agents: run the model on recordings of your own calls, upload a dictionary of drugs and doctors' names, and see how many errors disappear specifically on medications and numbers.
- Nov 18, 2026Devin Bot built an Android app on its own in 44 days: 1 134 PRs from 183 messages
- Nov 18, 2026Duco cuts agent context by 90% and onboards bank reconciliations 10x faster
- Nov 18, 2026Epic to remove Blueprints from Unreal Engine 6: AI works with code, not diagrams
- Nov 18, 2026rust-anything teaches AI agents to rewrite apps in Rust module by module
Key Questions Answered
What the model can do?
Scribe v3 Live Medical is built on the new Scribe v3 Live model and is already available in the ElevenAgents platform: open Automatic Speech Recognition in the settings and pick it from the list. Previously, ElevenLabs offered medical transcription only for recordings, such as dictations and visits. Now it works on live calls: appointment booking, information lines, phone insurance verification, and follow-ups on claims.
What the tests showed?
The company compared the model against streaming models from leading providers, both medical and general-purpose, on the same recordings. It chose PriMock57 because it contains everything that makes live calls hard: unscripted conversation, phone audio with background noise, and dense clinical detail. On Fareez OSCE (272 recordings of training patient interviews), the model has the lowest "semantic" error rate among medical models, at 0.6%.
What are the key details on Privacy and caveats?
The model meets HIPAA requirements for enterprise customers with a signed BAA and Zero Retention mode enabled (data is not stored). ElevenAgents holds SOC 2 Type II, ISO 27001 and PCI DSS Level 1 certifications. Patient data is stored in the US by default, and isolated environments in the EU, India or Singapore can be used if needed. A caveat: all comparisons were run by ElevenLabs itself, on providers it selected and in the best configuration for each.
Ship AI Faster with Founders, Engineers & Weekly Architecture Reviews
Inside Vibe Club (hamidun.com/club) — weekly live masterminds with Zhemal Khamidun, Tech Radar, full LMS access, and 370+ production Claude Code skills.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.