BARY Voice Assistant Moves to a 4 GB Controller and Trains in 47 Minutes
Over the course of a year, the author of the BARY home assistant moved it from an Orange Pi to a Wiren Board 8 controller with 4 GB of memory, which already runs his home. Training the model there initially froze the controller. The culprit was not the neural network itself but two extra data-loading processes that ate the memory. After the fixes, a full training cycle took about 47 minutes, with peak memory at 1.47 GB. The practical takeaway: before buying more powerful hardware, measure where the memory is going.
AI-processed from Habr AI; edited by Hamidun News
A year ago, the author of BARY built a local, cloud-free voice assistant for his smart home. Now he has reported what happened after moving it to new hardware. The key point: the entire assistant (speech recognition, command lookup, speech synthesis) now runs on a Wiren Board 8 controller with 4 GB of memory, and it is also fine-tuned there in 47 minutes. At first, that training simply froze the controller, and weak hardware turned out not to be the cause. Extra processes around the model were eating the memory. If you run AI on a low-power device, this is worth knowing: instead of buying a more expensive board, first measure what exactly is occupying the memory.
The home knows where it was called from
The phrase "Bari, turn on the light" is incomplete: which light, and in which room? So every microphone now has a room assigned to it. Say it in the kitchen and the kitchen light comes on; you don't need to speak the device's full name. The author's home has 49 devices in 18 rooms, so this simplifies speech considerably.
The best microphone turned out to be the Seeed ReSpeaker USB Mic Array v2.0, which readers recommended in the comments on the first article. It can be connected to the controller over USB or placed in a distant room together with a Raspberry Pi 3B+. That satellite recognizes speech itself via VOSK and sends the controller the text, recognition variants with confidence scores, and a wake word event. As a fallback, it can also send a WAV recording, which GigaAM processes in 240–370 ms. The system can support up to four such points, though only one is running in tests so far.
The word "Bari" got its own model built on OpenWakeWord. Only the classifier is trained; the neural network that produces the embeddings is a pretrained one. It needs about a hundred labeled fragments plus noise samples from the library's dataset. The author hunted for false positives with a TV: everything the model reacted to went onto an ignore list. The weak spot: the data is almost entirely the author's own voice, and the model does not trigger on his wife or child.
The command model itself has changed too. Previously, every combination of action, device, room, and parameters was a separate phrase, which is where the 16 thousand commands came from. Now the model has several outputs: action, device, room, value, and delay.
The phrase "Turn off the light in the study in five minutes" becomes a ready-made structure with the fields action, device, room, and delay. The model has about 6 million parameters (previously about 5 million) and a vocabulary of 603 tokens. New devices are pulled in from BARY along with their room and capabilities, and synonyms such as "telek" (slang for TV) are added without retraining: it is enough to recompute the index, and a full recomputation takes 9.
4 seconds.
Training on the controller: PyTorch wasn't what ate the memory
The controller is a Wiren Board 8.5.1 on an Allwinner T507 with 4 GB of memory. In one measurement, the BARY server used 275 MB, the assistant plugin 182 MB, the ML service 377 MB, VOSK 233 MB, and Piper 283 MB. That totals about 1.35 GB, though simply adding up RSS values gives only a rough estimate.
The first attempt to train the model on the controller ended with the system going into swap and freezing. Measurements showed that PyTorch's DataLoader launched two worker processes, each with its own copy of the data and libraries, while at the same time the assistant could be recomputing the command index. Here is what was fixed:
- on controllers with less than 6 GB of memory, a single process loads the data (num_workers=0);
- training and index recomputation no longer run at the same time;
- the memory monitor counts child processes, not just the main one;
- if MemAvailable stays below 128 MB for three checks in a row (one every five seconds), training stops and saves an intermediate result;
- the old model keeps running until the new one has finished training and passed validation.
The model, dataset, and batch size did not change. A cycle of 20 epochs with 64 examples each took about 47 minutes, the trainer used no more than 1.47 GB, and free memory never dropped below 799 MB. After training, the new model loaded on its own and built the command index in 9 seconds. A caveat: the latest run used only 343 training and 35 validation examples, the best model came out at the third epoch, and there is no accuracy threshold on an independent test set yet.
LLM in one place, lights in another
The controller handles lights, climate, and curtains on its own, without an LLM. A phrase is parsed in 30–70 ms, and the response to a short command arrives in roughly 600 ms, though 500 of that is a deliberate wait while the assistant makes sure the phrase has ended. The author does not want to put a generative model between "turn on the light" and the relay: the neural network only parses the phrase and hands BARY a ready-made command, and the wall switch works even without the assistant.
The LLM is needed only for free-form conversation ("Bari, let's talk") and for parsing unrecognized phrases. It runs through Ollama on an ASUS TUF Dash F15 laptop with an 8 GB RTX 3070 Laptop GPU, using the qwen3:8b model. A response takes about 12 seconds to generate. If the laptop is turned off, only conversation is lost, while lights and heating keep working. For questions about BARY itself, the assistant first finds the answer in its built-in documentation and passes it to the model, so the model reads the current text instead of recalling it from memory. The author handed speech output over to a smart speaker: Piper sounds bad, and Silero is better but still doesn't satisfy him.
Besides devices, the assistant can control music, a shopping list, and movie and TV series downloads. The practical lesson for developers of local systems is simple: keep critical actions outside the generative model, and before changing hardware, measure where it spends its resources.
- Oct 12, 2026CLEAT Teaches Qwen3-4B Agents to Ask for Clarification at the Right Time: Up to +13.2 pp
- Oct 23, 2026HoloBoxing teaches a humanoid to box in 7 minutes on a single GPU
- Jun 2, 2026Gemini in Android Auto: How a Voice Assistant Changed My Commute
- May 2, 2026Yandex Cloud explains how to deploy AI agents in security without false lockouts and outages
Key Questions Answered
What are the key details on The home knows where it was called from?
The phrase "Bari, turn on the light" is incomplete: which light, and in which room? So every microphone now has a room assigned to it. Say it in the kitchen and the kitchen light comes on; you don't need to speak the device's full name. The author's home has 49 devices in 18 rooms, so this simplifies speech considerably. The best microphone turned out to be the Seeed ReSpeaker USB Mic Array v2.0, which readers recommended in the comments on the first article.
What are the key details on Training on the controller: PyTorch wasn't what ate the memory?
The controller is a Wiren Board 8.5.1 on an Allwinner T507 with 4 GB of memory. In one measurement, the BARY server used 275 MB, the assistant plugin 182 MB, the ML service 377 MB, VOSK 233 MB, and Piper 283 MB. That totals about 1.35 GB, though simply adding up RSS values gives only a rough estimate. The first attempt to train the model on the controller ended with the system going into swap and freezing.
What are the key details on LLM in one place, lights in another?
The controller handles lights, climate, and curtains on its own, without an LLM. A phrase is parsed in 30–70 ms, and the response to a short command arrives in roughly 600 ms, though 500 of that is a deliberate wait while the assistant makes sure the phrase has ended. The author does not want to put a generative model between "turn on the light" and the relay: the neural network only parses the phrase and hands BARY a ready-made command, and the wall switch works even without the assistant.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.