Agent orchestration matters more than model choice, arXiv study finds

A study published on arXiv in July 2026 found something unexpected: optimizing the orchestration layer of AI agents (the way the system assembles context and manages tools, action sequences and execution logic) affects costs more than the choice of model itself.

How the orchestration effect was tested

The authors ran a controlled experiment using strict substitution. They selected 22 fixed agent evaluation tasks and tested them on six foundation models. Only the orchestration layer changed; everything else stayed the same.

On one side was a standard production architecture (the ordinary system used in real deployments); on the other, an optimized Writer Agent Harness (a purpose-built, improved orchestration layer). All models ran the same tasks, with the same duration and the same context makeup.

Results: numbers that surprise

• Cost reduction: $0.21 → $0.12 per task (41% savings)
• Faster execution: 48 seconds → 27 seconds (44% faster)
• Token savings: 14.2k → 8.8k tokens per task (38% fewer)
• Quality: 0.78 → 0.81 on the completion metric (maintained or improved)
• Quality per dollar rose by 82%
• Models tested: Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1, Palmyra X6

The most striking finding: on this workload, the orchestration layer affected task cost more than the full range of price differences between the models themselves.

"Orchestration is a single component whose efficiency multiplies

across every model an organization runs: current and future."

Does the effect hold across all models?

Yes. All six models became 33-61% cheaper, regardless of their base cost and performance. The effect is universal.

But quality gains were uneven: stronger models improved more than weaker ones. The authors called this "harness leverage", the leverage effect of orchestration. The correlation between quality improvement and a model's baseline capability was r=0.99 (a near-perfect relationship).

What mechanisms drive the savings

The authors identified six families of mechanisms that together produce the gain:

• Caching discipline: the right cache shape and size, avoiding cache misses
• Prompt structuring: the order in which context is supplied, prioritizing important data
• Error handling: if a tool call returns an error, don't blindly retry it
• Retry optimization: when to move to the next step and when another turn is needed
• Observability and logging: seeing what the agent does and catching unusual behavior early
• Failure cost management: not overpaying for unsuccessful attempts

Together, these mechanisms also multiply one another.

What this means for enterprise

Model choice is only part of the cost-optimization problem. Orchestration engineering (how context is assembled, how caching works, how the tool-calling chain is managed, how errors and failures are handled) has more leverage over economics than the choice of model.

For companies deploying AI agents in production, this means that before paying for a more expensive or more powerful model, it makes sense to look at the architecture of the system itself. Well-designed orchestration can deliver bigger savings in cost and speed than a model upgrade. And the gain scales: if you run six models, better orchestration benefits all six.