Japanese Startup Open-Sources Haru-4B, a Voice AI That Listens While It Talks and Needs 12 GB of VRAM
Most voice assistants work like a walkie-talkie: they wait for you to stop talking, then respond. Japanese startup higurashi AI has open-sourced Haru-4B, a model that listens to you while it is speaking, drops in an occasional "uh-huh," and can talk over you. It needs an Nvidia GPU with 12 GB of memory, the kind found in an ordinary gaming card, and on a MacBook with an M1 Max the latency is 65.2 ms. You can now run a voice bot locally, without the cloud.
AI-processed from X: @TeksEdge; edited by Hamidun News
Almost every voice assistant works like a walkie-talkie: you speak, it waits for silence, then it answers. You can't interrupt it, and it won't throw in the familiar "uh-huh" or "I see" in the middle of your sentence, so the conversation feels like trading voice messages. Japanese startup higurashi AI has open-sourced Haru-4B, a model that listens to you at the same moment it is speaking.
Running it takes an Nvidia GPU with roughly 12 GB of VRAM. That is the level of an ordinary gaming card, not an expensive server accelerator. According to a post on X, the model quickly took first place on Hugging Face trending across all categories.
If you build voice products, this is a way to test live dialogue on your own hardware, without a cloud API.
What "listens while it speaks" means
The technical term for this mode is full-duplex. A phone call works the same way: both participants can talk and hear at once. Haru-4B does not wait for you to finish your sentence. The model continuously processes incoming audio and decides what to do: keep talking, stop, drop in a short "uh-huh," or speak over the other person.
The developers highlight three notable abilities: overlapping speech, conversational acknowledgments ("yeah, yeah," "got it"), and support for nine core languages. Picture a call center or a language tutor. A live agent nods along to your words and doesn't stay silent while you explain the problem. For a classic voice bot that is nearly out of reach, because it waits for the end of your turn.
Under the hood is the Qwen3.5-4B language model, with audio components added. According to the authors, training used more than 40M hours of real speech. Of that, over 13M hours are dialogues in which the speakers talk at the same time. The speech data covers more than 160 languages. The model can also adapt to characters and voices: an audio sample is enough.
Numbers and hardware
The most practical part of the release is the hardware requirements. An Nvidia GPU needs roughly 12 GB of VRAM. The checkpoint (the file with the weights) in BF16 format weighs about 10.5 GB. For server-side GPU serving, the team suggests vLLM-Omni. Mac users get a native MLX implementation with custom Metal kernels.
On a MacBook with an Apple M1 Max chip, the developers report a median processing latency of 65.2 ms. The model receives a new audio frame every 133 ms. In plain terms: processing a chunk of audio takes about half as long as the chunk itself lasts. So, judging by these figures, the model can keep up in real time on a laptop. An important caveat: all the numbers come from the authors' claims and are retold in the post. The material contains no independent measurements.
The license deserves separate mention. Apache 2.0 covers both the weights and the inference code. That means you can embed the model in a commercial product without negotiating with an owner and without paying for every request to someone else's API.
What to do with it, and where the catches are
For developers of voice interfaces, this is a chance to test the approach without cloud contracts. It suits language trainers, game NPCs, in-car assistants, or smart speakers, where privacy matters and the voice should not go to someone else's server. For product managers, one more thing matters: how natural a conversation feels often decides whether people use a voice feature at all.
The limitations are also clear. This is a 4B-parameter model, and in reasoning quality it is unlikely to match large cloud systems. The post on X gives no data on how Haru handles complex questions, background noise, or multiple speakers. Support for nine core languages and training on data from 160 languages are not the same thing: quality in less common languages will need to be checked separately. The ability to interrupt and talk over you can be irritating if it is poorly tuned. Another risk is voice cloning from an audio sample: a product would need consent rules and protection against forgery.
The practical step is simple: download the checkpoint, run it on your own scenarios, and compare it with the familiar "speech recognition, language model, speech synthesis" pipeline. If conversational liveliness matters for your product and you already have a 12 GB GPU, the test will take one evening.
- Nov 20, 2026Nvidia Put $58.1 Billion Into Other Companies in Nine Months — 12 Times More Than a Year Ago
- Nov 20, 2026Four Hours on an RTX 5090 Cost 348 ₽, or From 1 730 ₽ With a One-Day Minimum
- Nov 20, 2026Nvidia Is Close to Buying Inception for $500 Million to Get Diffusion LLMs
- Nov 20, 2026Nvidia quits selling H200 in China, and new US rules threaten $44–62 billion in lost revenue
Key Questions Answered
What "listens while it speaks" means?
The technical term for this mode is full-duplex. A phone call works the same way: both participants can talk and hear at once. Haru-4B does not wait for you to finish your sentence. The model continuously processes incoming audio and decides what to do: keep talking, stop, drop in a short "uh-huh," or speak over the other person. The developers highlight three notable abilities: overlapping speech, conversational acknowledgments ("yeah, yeah," "got it"), and support for nine core languages.
What are the key details on Numbers and hardware?
The most practical part of the release is the hardware requirements. An Nvidia GPU needs roughly 12 GB of VRAM. The checkpoint (the file with the weights) in BF16 format weighs about 10.5 GB. For server-side GPU serving, the team suggests vLLM-Omni. Mac users get a native MLX implementation with custom Metal kernels. On a MacBook with an Apple M1 Max chip, the developers report a median processing latency of 65.2 ms. The model receives a new audio frame every 133 ms.
What to do with it, and where the catches are?
For developers of voice interfaces, this is a chance to test the approach without cloud contracts. It suits language trainers, game NPCs, in-car assistants, or smart speakers, where privacy matters and the voice should not go to someone else's server. For product managers, one more thing matters: how natural a conversation feels often decides whether people use a voice feature at all. The limitations are also clear. This is a 4B-parameter model, and in reasoning quality it is unlikely to match large cloud systems.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.