Three LLMs Follow Prompts for Stutters and Interruptions Poorly; Rules Do Better

Voice agents are entering commercial use, and they are tested on user simulators that are supposed to talk like real people: stumble, interrupt, and drop in "uh-huh" and "mm-hmm." The authors of the paper Prompts versus Rules checked whether such simulators do what they are told. They compared three LLMs with prompts for tau-Voice against their own rule-based algorithm, which does not use a model at all. Three types of behavior, two approaches, one conclusion: prompts work poorly.

What Exactly Was Tested

In the voice agent industry, the quality of an evaluation depends on the quality of the simulator. If the virtual "customer" speaks too smoothly or interrupts at random points, the agent's test results do not reflect real-world use. The researchers noticed that most earlier studies do not check whether the configuration that was set actually shows up in the finished simulation. A developer writes "add stutters and interruptions," gets audio back and considers the job done.

The experiment looks at three kinds of speech behavior:

• disfluencies, meaning stutters, repetitions and hesitations;
• interruptions of the agent's turn;
• backchanneling, short reactions such as "I see" and "uh-huh" while the other party is speaking.

The authors audited not the configured parameters but the behavior actually realized: what was ultimately heard, and exactly where.

Where Prompting Breaks Down

The prompt-based approach was tested on three models. The result was consistently poor: the speech does not match the instructions. In addition, stutters, interruptions and backchannels are placed and distributed less naturally than the instruction itself implies. A model can be told to behave "like a person who often loses their train of thought." But real pauses and interruptions depend on syntax, rhythm and the moment in the conversation, and an LLM without specialized training does not sense them.

The abstract gives no numerical deviation metrics, so the size of the gap cannot be judged from the abstract alone. The direction, however, is unambiguous. General-purpose LLMs that are not specifically trained to imitate users do not, on their own, reproduce genuine human behavior.

What Rules Delivered

The authors' alternative is a rule-based injection algorithm that uses no model. It inserts disfluencies, interruptions and backchannels according to deterministic, linguistically grounded rules. According to the study, this approach produces controllable yet varied behavior that is closer to natural speech. If a parameter is set, it is actually realized in the simulation. That is exactly what prompting lacks.

The authors' conclusion goes beyond a single algorithm. To close the gap, you need either linguistically grounded deterministic methods or specialized models for user simulation. At the same time, the community should publish realized values, not just settings.

How to Apply the Findings in Practice

Teams that test voice agents can take several steps right now:

• do not treat a simulation as natural just because the prompt specifies stutters and interruptions;
• measure the actual frequency and placement of these events in the generated turns;
• for controlled scenarios, combine an LLM that writes the content of the turn with a deterministic layer that applies the behavior;
• when publishing test results, report the realized figures, otherwise comparing agents against each other becomes meaningless.

For the voice agent market, this is a question of reproducibility. If the test "user" is unpredictable, the agent's quality metrics are unpredictable too. Simple rules look more modest than an LLM here, but they provide the control needed for a fair evaluation.