DialFork measured procedural hallucinations in agents across 36,278 dialogues

The Idiap team has published DialFork, the first benchmark that measures procedural hallucinations in dialogue agents directly from the raw text of a conversation. The dataset contains 36,278 English-language dialogues, 24 tasks, 12 domains, and 4 types of hallucinations. The data and code are released under the MIT license, and the project has a public leaderboard.

The paper's authors are Sergio Burdisso, Esaú Villatoro-Tello, Dairazalia Sánchez-Cortés, Yanis Labrak, Jordi Luque, Milos Cernak, and Petr Motlicek.

The agent is polite, the facts are correct, but there's no result

The authors explain the problem with a simple example. You ask an LLM agent to check the weather. It's courteous, articulate, and never says anything false. But twenty replies later, you still don't have a forecast. The agent has stopped following the procedure it was given.

That's a procedural hallucination. Fact-checkers don't catch it because the statements are all accurate. Turn-by-turn quality metrics miss it too, since each individual reply looks fine. The failure only shows up in the trajectory of the entire conversation.

The more agents get deployed for customer support, medical appointment booking, lead collection, and banking scenarios, the more expensive this kind of drift becomes. The scenario there is strictly defined, and deviating from it counts as a failure even if it sounds convincing.

How a fork is made and where the labels come from

DialFork's main insight is that nothing needs to be labeled by hand. You take a dialogue where the agent does everything correctly. You replay it with the same seed. At a chosen moment, you give the agent a new instruction, and the conversation "forks."

The fork's position immediately yields two labels. The first shows how far the branch has drifted from the reference scenario. The second indicates the turn at which the deviation began. For every dialogue, these labels are generated automatically.

The construction was verified manually: 30 annotators confirmed that the forks are indeed built the way they're intended to be.

Three tasks and two metrics that look at the flow of the conversation

The benchmark describes deviation at three levels of granularity:

• Set-to-set distance (monitoring). This evaluates how much a deployed agent's set of dialogues differs from the reference set. This task is useful for a team monitoring an agent in production.
• Dialog-to-dialog distance (procedural search). This evaluates how much a single dialogue deviates from the reference, and that distance can be used to search for dialogues with a similar procedure.
• Turn prediction (real-time guardrails). The task is to identify the turn at which the dialogue first went off the rails. This formulation is exactly what real-time guardrail mechanisms are built on.

For the first two tasks, the authors proposed their own metrics that account for the flow of the dialogue:

• FlowKL compares sets of dialogues.
• FlowScore compares individual dialogues.

Both metrics evaluate how the conversation unfolds, not what it's about. At the level of a single dialogue, FlowScore, according to the authors, significantly outperforms all baseline methods across seven different encoders. The conclusion is direct: semantic similarity between texts is not a substitute for procedural similarity. Two dialogues can discuss the same thing yet follow different paths, and ordinary embeddings won't catch that difference.

What developers should do with this

If you're building agents with a strictly defined scenario, DialFork offers three practical applications.

First: run your system through the 24 tasks across 12 domains and get a drift assessment that turn-by-turn metrics don't provide. Second: use FlowKL for monitoring. You can compare the distribution of live dialogues against the reference and catch drift before user complaints arrive. Third: use the turn-prediction task to train and test guardrails that interrupt or steer the agent back on script at the first deviation, not the twentieth.

The dataset is available on Hugging Face under the DialFork organization, and the code and leaderboard are in Idiap's repository on GitHub. The MIT license allows the materials to be used in commercial projects. For now the benchmark is English-only, so other languages will require reproducing the fork methodology independently. The procedure itself, though, is easy to replicate: you need a working dialogue, a fixed seed, and a point at which the agent is handed a new instruction.