Apple ML Research proposed compact models for correcting speech recognition errors

Researchers at Apple ML Research in the paper "Revisiting ASR Error Correction with Specialized Models" proposed replacing bulky large language models with compact seq2seq models for correcting errors in automatic speech recognition (ASR).

Why LLMs Are Not Ideal for Correcting ASR Errors

Most existing methods for correcting ASR errors rely on text models that know nothing about the specific patterns of speech recognition errors. Recently, large language models have begun to be used for this task, but they have two serious drawbacks: they add latency to the response and are prone to hallucinations — that is, they can "correct" the text in such a way that it no longer matches what was actually said in the audio.

How the Proposed Solution Works

The paper's authors returned to the idea of compact seq2seq models trained on ASR errors from real and synthetic audio recordings. To scale the volume of training data, they build synthetic corpora through a cascade of "speech synthesis → speech recognition" (TTS, then ASR again): the text is first pronounced by a speech synthesizer, and then the resulting sound is passed through a recognizer, which outputs realistic errors for training the correction model.

• Method — compact seq2seq models instead of large LLMs for ASR error correction
• Synthetic training data is generated through a TTS → ASR cascade
• Key quality factor — matching the diversity of errors to the real distribution, not just the volume of data
• A separate decoding approach is proposed — correction-first decoding

The key finding of the research is that the quantity of synthetic data matters less than how accurately the distribution of generated errors matches real speech recognition errors. The authors also propose a separate decoding strategy — correction-first decoding, in which the error correction stage is moved to the beginning of the processing pipeline.

What This Means

The work demonstrates a countermovement in the voice interface industry: rather than solving the ASR error problem by connecting increasingly large LLMs, Apple is exploring the path of narrowly specialized compact models — they are cheaper to infer and less prone to hallucinations, which is critical for voice assistants operating in real time on devices with limited resources.