EvoSyn: Multi-Agent Data Synthesis Lifts Task Success by 13.3 Points

The authors of arXiv preprint 2610.12593 have introduced EvoSyn, a multi-agent system that synthesizes training data for interactive LLM agents. According to the reported results, it raises average task success on the τ²-Bench and CAR-Bench benchmarks by 13.3 points over the strongest synthetic baseline tested. The system also finds controllable interaction factors that go unnoticed if the student agent's feedback is ignored.

The problem: errors are visible, decisions are not

Fine-tuning agents increasingly depends on synthetic experience. It is not enough to produce trajectories that execute and pass validation. They must also help the current agent specifically: cover its weak spots rather than repeat what it can already do.

Modern methods use environment feedback, data validation, and the failures of the model being trained. The authors call the remaining problem the feedback-to-decision gap. Suppose an agent fails on a certain class of scenarios. The failure alone does not say how to change the next batch of synthetic data. It also leaves open another question: whether the current generator can produce such scenarios at all, or whether the needed axis of variation is missing from its space.

How EvoSyn works

The central object of the system is a mutable synthesis state. It records which interaction factors can be generated in a targeted way and where effort should go. This is not a static set of prompts but a structure that is revised after each training cycle.

A multi-agent system with separated roles handles the revisions. Some agents translate the student's feedback into synthesis edits. The Designer allocates generation directions. The Planner turns them into executable experience, meaning concrete trajectories.

The process runs in two modes:

• adaptation within the space: shifting generation toward weak and under-covered areas;
• expansion of the space: new factors discovered along the way become explicit generation control parameters.

The second mode solves a problem that fixed templates cannot see. If the needed axis of variation is absent from the description, no amount of reweighting will make the required scenarios appear.

What the experiments showed

The abstract reports the following results:

• +13.3 points in average task success over the strongest synthetic baseline;
• testing on τ²-Bench and CAR-Bench, benchmarks for interactive agents;
• higher quality and structurally broader coverage of synthetic experience;
• discovery of controllable factors that are missed without student feedback.

The limitations are also clear. The abstract gives no absolute metric values, base model sizes, or generation costs. The work remains a preprint without peer review, and reproducibility and the contribution of each component (Designer, Planner, space expansion) will have to be checked in the full text.

Where this is useful

For teams building agents with tool calling and multi-turn dialogues, the idea is clear and portable. Data should be synthesized based on the evaluation results of a specific model, not "in general." The practical minimum looks like this: keep an explicit description of the factors you use to generate scenarios, and regularly check it against the agent's failure logs. If failures cluster around a parameter that the generator does not have, that is a signal to expand the space, not just to increase volume.

Before adopting the approach widely, check three things. Does the gain hold on your own domains, not just on two benchmarks? How much does the extra loop of several LLM agents cost compared with ordinary synthesis? Is the agent overfitting to the tests? The answers will be in the details of the experiments, and that is where to look first.