The Trinity improved an AI agent’s harness, adding 9.8 points on new tasks

Your agent enters a room it has already checked for the second time, forgets a constraint and burns its remaining budget on useless calls. The immediate temptation is to buy access to a stronger model. But first, it is worth checking the code around it: what gets stored in memory, which observations make it into the prompt and what decides when to stop. The Trinity explores how to improve this very layer automatically.

According to the results presented in the material, the first iteration’s modules raised the average score on new tasks from 37.45 to 47.26 — roughly 9.8 points. That is a useful result without changing the model’s weights. The more striking 26.2-point gain also appears in the tables, but it includes additional conditions that substantially change what the comparison means.

The agent writes its own operating instructions

The harness is the code that turns a model into an agent capable of taking action. It prepares observations, stores history, coordinates tool calls, checks constraints and manages retries. Even a good model will struggle if this code keeps losing an important part of its state.

Imagine a maze with gates: you can move forward, but you cannot return along the same path. Finding a short route is not enough. The agent needs to remember where the gates are and check the return route in advance. In The Trinity, an error like this becomes a reason to add a corresponding module to the harness.

The system divides the work among three roles performed by a single frozen model. Evolver analyzes errors and proposes changes. Creator builds simplified training environments. Explorer works through them and collects trajectories: sequences of observations, decisions and outcomes. A separate software host validates the environments and proposed changes.

Tasks are grouped by the capabilities they require. A maze and a visual puzzle may end up in the same group if both require accurately interpreting observations and remembering state. For these groups, the system creates small worlds where the relevant error is easier to reproduce and analyze than in a long, full-scale game.

Why the 26-point gain needs explanation

After four iterations, the average score across 18 new benchmarks rose from 37.45 to 63.61 points. However, the upper estimate uses an ideal choice of harness: each task gets the best variant. The actual router, which has to choose in advance based on the available description, scored 60.96 points.

There is also preparation before evaluation. Each harness gets five attempts on the corresponding benchmark, after which that experience is distilled into stored skills. These preparatory episodes do not count toward the final score. The result therefore cannot be read as solving entirely unfamiliar tasks on the first encounter.

The first-iteration comparison helps separate the effects. The learned modules alone add about 9.8 points over the baseline, while the prepared skills alone add about 11.2. Together, they add about 16.1. The author of the analysis did not find an equivalent comparison without preparation for the fourth iteration.

The two physical robots in the new section also require a separate qualification: the modules were trained on simulated versions of those same tasks. What is being tested here is transfer from simulation to hardware. And the 60% result means six successful episodes out of ten — the experiment is still small in scale.

What to apply to your own product

The most interesting comparison concerns the training environments. When training directly on benchmarks, the capability tree barely improved performance over a single shared harness, but required roughly three times fewer tokens. Switching to self-generated simulations added almost five points on new tasks, although it increased training costs.

For a team, this is a reason to start with a log of recurring failures. If the agent regularly forgets the conditions for returning, create a small environment with those conditions. If it violates a tool’s constraints, build short, verifiable tasks around that failure. Then compare the modified harness with the original on held-out scenarios under the same budget.

The Trinity offers a practical hypothesis: some agent limitations can be addressed through memory, checks and training tasks. But the reported gain should be broken down into its components: how much came from the modules themselves, how much from preliminary attempts and how well the system selects a suitable harness in an actual run.