StopLoss Cuts Off Doomed Coding-Agent Runs and Saves Up to 48% of Costs

Coding agents keep stubbornly refining solutions that will fail verification anyway, and every extra step costs tokens. Handing the oversight to a frontier model runs into the same problem: the observer itself is expensive. The authors of StopLoss propose a small external monitor that reads the agent's last steps and stops the run in time. On SWE-bench Verified, it lowers the cost per saved solution by 4.5% while retaining 99% of successful runs. On the costlier SWE-bench Pro subset, savings reach 31%, and with Qwen3-8B as the base model they reach 48%.

How the monitor works

The name comes from the stock-market stop-loss order, which closes a position automatically when a loss becomes more likely than a profit. Here, the "position" is an agent run. The monitor decides to abort when its estimated risk of failure exceeds a threshold.

Training uses already completed trajectories and verifier verdicts. The outcome of the whole run, success or failure, is assigned to earlier local windows: the last few tool calls. This produces a training set without manual labeling. A fine-tuned language model scores each window in a single forward pass, with no long reasoning and no text generation. The threshold is calibrated on successful runs so that the monitor does not interrupt those that would have reached a solution. The executor stays untouched: the monitor plugs in from outside and does not require changes to the agent or its prompts.

The model was trained on GPT-family trajectories from SWE-rebench. It was tested on SWE-bench Verified and a subset of SWE-bench Pro, with both GPT and Claude serving as executors. The baseline was six general-purpose LLM judges, which were given exactly the same windows.

What the measurements showed

For general-purpose judges, failure prediction on the first check is close to random. Training on outcomes lifts it to a useful signal. In addition, each check costs one to two orders of magnitude less than with any of the tested judges. That is what makes early stopping economically sensible: if the check itself is expensive, it eats up the gains from aborted runs.

• Cost per saved solution on Verified: −4.5% relative to running all tasks to the end, with 99% of successful runs retained.
• On the Pro subset: up to −31%, and up to −48% with a Qwen3-8B base.
• Cost of a check: one to two orders of magnitude lower than with any of the six judges.
• In all four configurations, StopLoss's cost per saved solution is lower than that of any judge-based policy.
• The executor does not change, and the threshold is tuned on successful runs.

The difference between the benchmarks is telling. Verified tasks are cheaper, so there is little to save: 4.5% is a modest figure. On Pro, where a single failed run costs noticeably more, aborting failures pays off more. The effect grows with the price of a mistake.

Where this is useful in practice

The "cost per saved solution" metric is more useful than cost per run. It accounts both for the tokens saved and for the successes lost to false stops. Teams that run agents at scale, for example on automatic bug fixing in CI, should track this metric.

The approach does not require access to the executor's internals and relies on data most teams already have: trajectory logs and test results. A small monitor can be fine-tuned on a team's own runs, with the threshold set to an acceptable share of lost successes. In the paper, that share is 1% on Verified.

There are limitations, too. The results come from two software-engineering benchmarks, and the training trajectories belong to the GPT family. The paper does not show how the monitor would behave on other domain workloads or with other agent harnesses. Still, the idea is clear: if failure can be recognized cheaply, there is no need to wait for the end of a run.