Yandex: Alice processes smart-home commands on the speaker itself — on average 6x faster than the cloud

• Станция Миди и голосовое управление Zigbee-устройствами без интернета. История разработки — Habr (блог Яндекса), 2023-11
• Итоги Яндекса за 2025: 5,3 млн активных устройств YaOS/YaOS X, 2,9 млрд запросов к Алисе — Habr, 2025
• Яндекс.Станция — линейка устройств, включая Станцию Мини 3 Про с локальной обработкой команд — TAdviser, n/a
• Яндекс Станция Миди — официальная страница продукта (Zigbee-хаб, локальные команды) — Яндекс, n/a

Background

Alice is Yandex's voice assistant living in smart speakers and TVs on the YaOS and YaOS X platforms. The ecosystem's scale as of year-end 2025: 5.3 million active devices, 2.9 billion requests processed over the year, and 14 new smart gadgets in the lineup (Yandex data published on Habr). It is one of the largest voice ecosystems outside the English-speaking world.

In November 2023 Yandex released Station Midi — the first speaker in which Alice learned to understand and execute smart-home commands locally, right on the device, without a round trip to the cloud. Midi also acts as a Zigbee hub itself: sensors, bulbs, and plugs connect to the speaker directly, bypassing cloud services.

What makes the case notable engineering-wise: Yandex is a rare company that publicly detailed its on-device architecture — the team described it in a technical Habr post in November 2023, with concrete model sizes, memory footprints, and the history of optimizations. For comparison: Amazon and Google also moved voice-command processing onto devices, but never provided such detailed public engineering documentation of their solutions.

That openness makes the case verifiable and instructive: you can trace how a server-side speech recognition and understanding stack occupying tens of gigabytes gets squeezed into a speaker with one gigabyte of RAM.

Problem

The classic voice-assistant architecture routes every command through the cloud: the voice recording goes to a server, is recognized and interpreted there, and the response returns to the device. For a smart home this means two problems. First, the latency between saying 'turn off the light' and the light actually going off: even with good internet the command makes a full network round trip, although the speaker and the bulb are in the same room. Second, a completely non-functional smart home during internet outages: a light switch that dies with the Wi-Fi undermines trust in the very idea of a smart home.

Moving processing onto the device ran into hard resource arithmetic. Alice's server stack was designed for a data center: the cloud NLU service 'Begemot' alone requires at least 30 gigabytes of RAM and about 30 gigabytes of disk. A typical smart speaker (Station Mini) has 256 megabytes of RAM and 256 megabytes of flash — a hundred-plus-fold gap. On-device speech recognition also has to work in streaming mode — starting to parse the phrase before the person finishes it, otherwise there is no speed win.

So the task was framed not as 'put a model on a speaker' but as a full engineering program: rebuild three subsystems — speech recognition (ASR), language understanding (NLU), and the smart-home backend — into versions that fit within single and double-digit megabytes, without quality loss on the target class of commands.

Solution

The foundation was laid at hardware design time. Station Midi is built on an Amlogic A113X2 SoC with a built-in neural processor (NPU), 1 gigabyte of RAM, and 8 gigabytes of flash — versus 256 megabytes of each in the Mini. The NPU and the memory headroom were a deliberate bet on local models, made before those models were ready.

Speech recognition (ASR) was built on a transformer encoder with a move from CTC to RNN-T — an architecture enabling streaming, real-time recognition: the speaker parses the phrase as it is being spoken. The model, roughly ten million parameters, was trained on the same dataset as the cloud recognizer, with smart-home command examples added. For comparison: cloud ASR models are orders of magnitude larger.

Language understanding was solved by adapting the server-side 'Begemot' service — thus 'Begemotik' was born. From a service requiring at least 30 gigabytes of RAM, everything unnecessary for the smart-home domain was cut: heavy ML components, geography and date recognition. The result — 90 megabytes of RAM and 73 megabytes of flash: two-to-three orders of magnitude of compression while preserving quality on the target commands.

The third component is the local smart-home backend in Go: the command-execution logic that consumed 500+ megabytes in the cloud was squeezed to ~200 megabytes of RAM and 90 megabytes of flash — partly by dropping speech synthesis and keeping local smart-home configuration storage with syncing.

The connecting link is the built-in Zigbee hub: smart-home devices connect to the speaker directly, so the whole path 'voice → recognition → interpretation → device command' stays inside the apartment. The internet is not needed for these scenarios at all. The architecture is hybrid, though: anything beyond routine smart-home commands — music, search, conversation with Alice — still goes to the cloud, where the full-size models run.

Notably, the team published not just the result but the development history with intermediate failures and forks (for example, why CTC was abandoned in favor of RNN-T) — a rare level of engineering transparency for a consumer product.

Result

According to the Yandex team, local processing of smart-home commands is on average 6x faster than the cloud path — the gain comes from eliminating the network round trip plus streaming on-device processing; the company published no exact milliseconds, and recognition speed varies with phrase complexity. Lights and other Zigbee devices keep working when the internet goes down — offline resilience became not a side effect but the project's second headline result.

The approach stuck across the lineup: newer devices, including Station Mini 3 Pro, also got local voice-command processing (TAdviser) — Midi was not an experiment but a shakedown of the architecture for the whole platform. The ecosystem kept growing meanwhile: 5.3 million active YaOS/YaOS X devices and 2.9 billion Alice requests in 2025.

The compression scale the team documented in numbers is telling in itself: NLU — from at least 30 gigabytes of RAM down to 90 megabytes; the smart-home backend — from 500+ to ~200 megabytes; ASR — down to a ~10-million-parameter model, orders of magnitude smaller than cloud counterparts. These figures are a rare public reference point for anyone assessing the feasibility of on-device AI on cheap hardware.

In our view, the case's main transferable lesson is the correct decomposition of the hybrid: Yandex did not try to drag 'all of Alice' onto the speaker; it isolated the narrow domain where locality yields the most value (routine smart-home commands — high-frequency, short, with a limited vocabulary) and left everything else to the cloud. That framing — 'local for what is frequent and simple; cloud for what is rare and complex' — applies far beyond the smart home: from bank voice menus to automotive assistants.

The second observation: on-device AI starts with hardware. The NPU in the SoC and the gigabyte of memory were designed into Midi upfront — on a weak chip this architecture would not have happened. Teams planning local models in devices should budget compute headroom a generation before the models are ready — otherwise, by the time the software matures, the hardware in devices already sold will not cope.

Lessons learned

1. Latency is a smart-home product metric: users feel the 'on average 6x' gain with every flick of a switch.
2. On-device AI is above all a compression discipline: a ~10M-parameter ASR and an NLU squeezed from 30 GB to 90 MB suffice for routine commands — no giant model required.
3. Compress by cutting the domain, not just by quantization: 'Begemotik' got hundreds of times smaller than 'Begemot' because everything the smart home doesn't need was cut out (geography, dates, heavy ML components).
4. Decompose the hybrid by frequency and complexity: local for frequent simple commands, cloud for rare complex queries; don't try to drag the whole assistant onto the device.
5. Offline resilience is a value of its own: a smart home that dies with the Wi-Fi undermines trust in the whole category.
6. Specialized hardware (an NPU in the SoC, memory headroom) must be planned at device-design time — a generation before the models are ready; you can't retrofit on-device models onto a weak chip.
7. A public engineering write-up (Habr) with concrete model sizes and a history of failures is a rare, correct practice: it makes the claimed results verifiable.

Frequently asked questions