STMicroelectronics' STM32N6 Runs Local Speech Recognition Without the Cloud at 0.2 W

Microcontrollers with a built-in NPU are starting to move into territory that used to belong almost entirely to cloud speech recognition services. An experiment on the STM32N6 showed that local recognition of free-form speech can already run directly on the device, with no internet, in near real time and at a power draw of about 0.215 W.

How the system works

The project's author built the speech recognition pipeline from three blocks: an acoustic model, a decoder and rescoring. The heaviest part is the acoustics: it takes the raw audio signal from the microphone and turns it into a sequence of phonemes. The decoder is supposed to assemble words from those phonemes, and the rescoring block to re-check the result in context. At this stage, what already runs on the STM32N6 is the acoustic model, which is the key foundation of the whole system.

In practice it looks like this: the device listens to speech in real time, runs it through the NPU and outputs a stream of phonemes. In the demo, words and numbers appear at the top, and the "raw" phonemes predicted by the model appear at the bottom. For now, phonemes are converted to words through rigid mapping rather than a full language decoder. That keeps the system limited, but the fact that the acoustic model runs locally on a microcontroller already matters more than the current scaffolding around it.

Numbers and limitations

The strongest result is power consumption. During active recognition, the whole system draws about 215 mW. Of that, roughly 160 mW goes to the NPU and the Cortex-M55 core, another 45 mW to external Flash and PSRAM, and about 10 mW to external pins. And this is not an optimized mode: the core does not yet use aggressive sleep, and the NPU is only 10.4% loaded, so there is still room to cut consumption further.

Quality also looks serious for hardware of this class. The model has 8.5 million parameters, lost almost no accuracy after quantization to int8, and on the target device reached a PER of 5.3% on dev_clean and 14.4% on dev_other. NPU inference took 52 ms per 500 ms of audio, and total latency was 985 ms. Almost half of that delay comes not from the hardware but from a "look-ahead window" the model uses to predict phonemes more accurately.

• Acoustic model size: 8.5 million parameters
• Power draw during recognition: about 0.215 W
• NPU inference time: 52 ms per 500 ms of audio
• Quality loss after int8 quantization: less than 0.5%
• RAM usage: 18%, NPU load: 10.4%

The comparison with larger systems matters separately. On PER, this model came out comparable to wav2vec 2.0 Base and HuBERT Base, even though those are roughly 11 times larger and are not designed to run on microcontrollers. The author is also candid about the project's limits: this is not yet a replacement for full text dictation, but rather a local engine for short commands and phrases where autonomy and energy savings are critical.

Where the microcontroller wins

The strength of this approach is not universality at any cost but filling the gap between simple keyword spotting and heavy cloud ASR. Ordinary local voice interfaces wait for an exact command match, whereas here the device can already interpret different phrasings of the same request. Instead of one rigidly defined phrase, the user can say "make it warmer," "raise it by five degrees" or "temperature up", and the system will understand a single action.

That opens up fairly practical scenarios: a smart home that does not send voice outside, voice entry of numbers and parameters in production, and work in warehouses, medical devices and vehicles, where the network is unstable or absent altogether.

Another plus is headroom. The STM32N6 currently uses only 18% of its RAM, and the NPU is used at about a tenth of its capacity. The next steps are clear: add a phoneme decoder, a language model and noise suppression. These are what should turn a convincing technical prototype into a genuinely useful user interface.

What this means

The STM32N6 does not replace cloud speech recognition, but it shows that some tasks can already be moved to the network edge with confidence. Where autonomy, privacy, cost and low power consumption matter, MCUs with an NPU are starting to look not like an experiment but like a new practical class of voice interfaces.