LangChain: claude-3.5-sonnet, gpt-4o, o1, and o3-mini Tested as ReAct Agents

The LangChain team published research in July 2026 called Benchmarking Single Agent Performance, in which they tested how the increase in the number of instructions and available tools affects the quality of work of a single ReAct agent.

What was tested

ReAct (Reasoning + Acting) is one of the basic patterns of building AI agents: the model alternately reasons aloud and calls tools, relying on the result of each previous step. The more tools and rules a developer adds to the system prompt of such an agent, the more complex the task becomes for the model — it needs to hold not only the current goal in the context, but also a growing list of available functions and constraints.

• The benchmark included models claude-3.5-sonnet, gpt-4o, o1 and o3-mini
• Tests were run in two different task domains
• The key variable of the experiment is the number of instructions and tools available to the agent at the same time

Why this is important for agent developers

In practice, teams building production agents often take the path of least resistance: they add more and more tools to the agent — database search, email sending, calendar work — without checking how this affects the quality of the model's decisions. Results of such benchmarks help understand if a particular model has a limit, after which the increase in the number of available functions starts to not help, but hinder the agent from choosing the correct next step.

Comparison of several model families at once — closed-source claude-3.5-sonnet and gpt-4o, as well as reasoning models o1 and o3-mini from OpenAI — shows that architectural differences between "regular" chat models and models designed for step-by-step reasoning can manifest differently precisely on tool-loaded agent scenarios, not on simple question-answer benchmarks.

What the comparison of domains revealed

Running the benchmark in two different task domains at once — rather than in one — allows you to separate the general pattern ("the more tools, the harder it is for the agent") from specific effects of a particular domain. If a model consistently loses in quality with an increase in the number of tools in both domains, this is a strong signal of a fundamental limitation of the ReAct approach as such, not just features of one task.

For engineers designing multi-agent systems, this is also an argument in favor of decomposition: instead of one agent with dozens of tools, it is often more effective to divide the task between several narrowly specialized agents, each working with a small, carefully selected set of functions.

What gives a comparison of four models at once

Simultaneous testing of claude-3.5-sonnet, gpt-4o and two OpenAI reasoning models — o1 and o3-mini — on the same set of tools and instructions gives developers a reference point when choosing a base model for their own agent: instead of relying on general question-answer benchmarks, they can rely on data about model behavior precisely in a tool-loaded agent scenario, which is closer to their real production task, where the tool count is in the tens, not units.

What this means

For engineering teams designing their own ReAct agents, the conclusion is practical: the choice of model and the limit on the number of simultaneously available tools is not a minor detail, but a parameter that is worth testing empirically on your own tasks, rather than assuming by default that "more tools — means the agent is smarter".