LangChain moved a coding agent from outside the top 30 to the top 5 on Terminal Bench, without changing the model
LangChain moved its Deep Agents coding agent from outside the top 30 to the top 5 on Terminal Bench 2.0 without changing the model. The team only reworked the harness around it. The baseline of 52.8% on GPT-5.2-Codex rose after three fixes: checking the solution against the tests before submission, giving the agent hints about its environment and time limits, and protection against getting stuck in loops. If an agent makes mistakes, the advice is to analyze its traces and refine the harness first, and only then change the model.
AI-processed from LangChain Blog; edited by Hamidun News
When a coding agent performs poorly, the first instinct is to reach for a stronger model. The LangChain team tested a different path: it kept the model as it was and reworked everything around it. Its Deep Agents agent climbed on Terminal Bench 2.0 from just outside the top 30 to the top 5 of the leaderboard. They changed only the harness: the system prompt, the tools, and the intermediate hooks around model and tool calls.
How they looked for weak spots
Terminal Bench 2.0 is a standard suite for testing programming agents: 89 tasks spanning machine learning, debugging, biology, and other areas. The baseline agent with GPT-5.2-Codex scored 52.8%. That is respectable, but below the top 30.
Next, LangChain built a repeatable improvement loop. All agent actions are written to LangSmith traces along with latencies, token counts, and costs. The trace analysis was packaged as an Agent Skill: it downloads the traces from a run, launches parallel agents that analyze the errors, and a main agent collects their findings into proposed edits. A human can review and discuss the changes, but that is optional. There is also a caveat: fixes tailored to one task can break others, so generalization has to be done carefully. The team promises to publish the skill.
What actually improved the results
Self-verification. The most common failure looked like this: the agent writes a solution, rereads its own code, decides everything is fine, and stops. They added a work order to the prompt: study the task and the code, make a plan that accounts for verification, write tests (including edge cases, not only the happy path), run them, read the full output, and compare it against what the task asked for rather than against the agent's own code. In addition, they built PreCompletionChecklistMiddleware, which intercepts the agent before it finishes and forces it to check the result against the specification once more.
Context about the environment. The agent does not know the directory structure, the installed tools, or the hard timeouts in advance. So at startup it is immediately given a map of the directories and a list of available tools, which reduces lookup errors. The agent is also told that programmatic tests will check the solution, and that file paths from the task must be followed literally. They also added warnings about the time limit, since agents estimate time poorly and fail to reach verification in time.
Escaping loops. Having settled on a plan, the agent sometimes makes 10 or more minor variations of an approach that is known not to work. The middleware counts edits to each file and, after a certain number, adds a prompt to reconsider the strategy. This does not always help: if the model is confident it is right, it may continue down the same path.
Reasoning budget. A maximum budget on every step eats up time and runs into timeouts. LangChain chose a "sandwich" scheme: xhigh for planning, high for implementation, and xhigh again for the final verification. In trial runs there was no noticeable difference between other ways of distributing the budget, so they settled on this one. In a multi-model setup, the same logic suggests assigning planning to a large model and implementation to a small one.
What to take from this
First, traces are a source of feedback. A model often goes the wrong way because it lacks the right tool or instruction, and both the prompts and the tools need to be examined together.
Second, guardrails such as loop detectors and test reminders are a temporary measure aimed at today's model weaknesses. The authors themselves write that as models improve, such crutches will likely become unnecessary.
Third, the harness has to be tuned to the specific model. A test with Claude Opus 4.6 on an early harness version was competitive but trailed Codex; the improvement loop was not run for Claude. General principles (context preparation, emphasis on verification) carry over, but several iterations tuned to your own model and task still pay off.
There are limitations too. The result comes from a single benchmark with hard timeouts, and real-world work usually has no such limits. The time-related prompts and some heuristics will therefore need to be adapted. Next, LangChain plans to explore combinations of several models (Codex, Gemini, Claude), memory for continual learning in agents, and ways to measure the effect of harness edits across different models. Deep Agents is an open-source project.
- Nov 21, 2026FDE job postings grew 4.6x in a year: what a forward deployed engineer actually does at the client's site
- Nov 21, 2026Sakana AI Opens Applied Research Lab for Market and Economic Modeling
- Nov 21, 2026Meta Is Reportedly Preparing the $300 Muse Charm AI Tamagotchi With a Year of Verizon 5G
- Nov 21, 2026Dreamteam Raises $53M for a CRM That Fills Itself from Email and Calls
Key Questions Answered
How they looked for weak spots?
Terminal Bench 2.0 is a standard suite for testing programming agents: 89 tasks spanning machine learning, debugging, biology, and other areas. The baseline agent with GPT-5.2-Codex scored 52.8%. That is respectable, but below the top 30. Next, LangChain built a repeatable improvement loop. All agent actions are written to LangSmith traces along with latencies, token counts, and costs.
What actually improved the results?
Self-verification. The most common failure looked like this: the agent writes a solution, rereads its own code, decides everything is fine, and stops. They added a work order to the prompt: study the task and the code, make a plan that accounts for verification, write tests (including edge cases, not only the happy path), run them, read the full output, and compare it against what the task asked for rather than against the agent's own code.
What to take from this?
First, traces are a source of feedback. A model often goes the wrong way because it lacks the right tool or instruction, and both the prompts and the tools need to be examined together. Second, guardrails such as loop detectors and test reminders are a temporary measure aimed at today's model weaknesses. The authors themselves write that as models improve, such crutches will likely become unnecessary. Third, the harness has to be tuned to the specific model.
Build & Ship Production AI Agents Without Boilerplate Code
Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.