Heartbeat Thinking Teaches Voice Agents to Work While You're Still Talking: Latency Down 36.4%

Voice agents with tools usually behave the same way: the user finishes speaking, and only then does the pause begin. The model assembles arguments, calls an API, waits for the response, and formulates a reply. A new paper from Hugging Face Daily Papers, Heartbeat Thinking, tries to claim that time back. On the FDBench v3 benchmark, the approach raised tool coverage by 16 percentage points and cut average final-response latency by 36.4% compared with a SHANKS-style baseline.

Why speculative launching breaks mid-sentence

The idea of starting work before the utterance ends is not new. If the first few words already make clear that the user needs a flight search or an order check, the agent can start early. The problem lies in how such systems are built. Speech is typically cut into windows or chunks, and generation is interrupted on every transcript update. As a result, a tool call can be cut off before its arguments are fully assembled. Compute time ends up tied to the rhythm of speech recognition, and less useful work gets done in advance than could be.

Consider the request: "Find me a flight to Istanbul for Friday... and also check whether I need a visa." The city and date come in the first half of the sentence. A window that restarts on every new word risks discarding a call that has already begun, even though it had enough information.

How the "heartbeat" works

The authors change how updates are handled. A new piece of speech does not cancel the current work; it launches a separate Thinker call. Calls already in progress and tools already dispatched are left to finish. The Thinker starts periodically, like a pulse, hence the name. Each run reads two sources: the latest partial transcript and a shared record of completed reasoning and tool results.

The shared record does two jobs. First, it can fill in missing arguments: if an earlier step already established that the user is talking about a specific flight, the next call does not start from scratch. Second, it makes it possible to launch dependent calls, meaning those that need the result of a previous tool. Preparation moves forward from one speech update to the next instead of resetting each time.

What the tests showed

The system was compared with SHANKS-style windows on three datasets: SpeechFCEval, FDBench v3, and AudioCRAG. The metrics differ. For function-calling tasks, the authors measured speculative tool coverage; for AudioCRAG, where retrieval matters, they measured reuse of retrieved results.

• The method increases aggregate tool coverage or retrieval reuse relative to the window-based baseline.
• FDBench v3: tool coverage +16 pp.
• FDBench v3: average final-response latency −36.4%.
• The impact on final task quality depends on the benchmark: the gain is not equally present everywhere.

The last point matters. The authors do not claim that the scheme always improves answers. The main gain lies in speed and in the share of work completed before the utterance ends. Quality mostly does not suffer, although the size of the effect varies across datasets.

What to change in production voice agents

For teams building voice assistants for call centers, cars, or smart speakers, the takeaway is practical. Latency after the end of an utterance usually consists of sequential steps, and a significant part of it can be shifted into the time while the person is still speaking. This takes three things: continuous reading of the partial transcript, a shared memory of intermediate results, and a rule against cancelling already launched calls without good reason.

There is a cost. Speculative launches spend more tokens and trigger more tool calls, and some of that work will turn out to be unnecessary if the user changes their mind mid-sentence. For tools with side effects, such as making a payment or sending a message, the scheme would have to be restricted to safe, read-only calls. But if the agent mainly looks up data and verifies facts, listening stops being idle time and becomes preparation of the answer.