LEASH hands hard steps to a strong agent and cuts Opus 5 costs by 19.2%

The LEASH authors propose a way to avoid running an expensive model on every step of a computer-use agent. A weak open model acts by default. The strong model switches on only when the weak one "doubts" itself, and it stays under supervision after taking over. On OSWorld, the pairing with Qwen3.5-35B-A3B completed 73.66% of tasks at $0.4117 per task. Claude Opus 5 running continuously scored 72.90%. That means LEASH matches it on quality and costs 19.2% less.

The problem: long tasks and expensive inference

Agents that control a computer solve tasks made up of dozens of steps: open an app, find a file, change a setting, check the result. A frontier model on every step quickly inflates the bill. A small open-weight model often fails to finish the job on its own. Weak-to-strong escalation promises a compromise, but it hinges on one question: when exactly should the strong model be called? Answering it requires a signal that can be checked on every step at almost no cost.

There is a second problem. The strong agent is usually accessed through a closed API, so its internal states cannot be inspected. Even after taking over, it can get lost, and there is nothing to monitor it with.

How LEASH's three layers work

The framework needs no training and relies on the weak model's hidden states. The authors compare them with states from past successful and failed trajectories. The more the current state resembles the failed ones, the higher the uncertainty score. The estimator reuses activations the model already computes when generating an action. No extra forward pass and no separately trained monitor are needed.

This signal feeds three layers:

• initial task uncertainty: early routing that decides whether the strong model is needed from the first step;
• step-level uncertainty: escalation during execution, when the weak agent starts to drift;
• shadow observer: keeps watching the strong black-box agent after it takes over.

The name LEASH describes the idea quite accurately: the strong agent is let off the leash when needed, but never released entirely.

What OSWorld showed

The experiments ran on OSWorld with four current backbones. Results as reported in the paper:

• LEASH consistently outperforms standalone weak agents;
• at the same escalation rate, it beats random escalation by 6.55–11.94 points;
• Qwen3.5-35B-A3B + LEASH: 73.66% success at $0.4117 per task;
• Claude Opus 5 in always-on mode: 72.90%;
• inference savings relative to always-on: 19.2%.

From the last item, one can estimate that a full run on Opus 5 costs about $0.51 per task. This is our own calculation; the paper's abstract does not give that figure. The 0.76-point accuracy gap on a single benchmark is best not overstated. But the direction is clear: smart routing catches up with the flagship on quality and wins on price.

Where it helps and where the catch is

For teams building agents on top of frontier-model APIs, the scheme looks practical. A local or cheap open-weight model does the bulk of the work, and the expensive call is reserved for hard spots. The longer the scenarios, the bigger the effect: the bill grows with the number of steps, while escalation triggers rarely.

There are limitations too. The method requires access to the weak model's hidden states, so it works only with a self-hosted or open-weight model. It also needs a library of past trajectories labeled as successes and failures, even though nothing has to be trained. Validation is so far limited to OSWorld, and how the signal behaves on other interfaces and in non-standard applications is unknown. Before bringing LEASH into production, it is worth collecting your own set of trajectories and checking how well the uncertainty signal agrees with real failures in your tasks.