Open GLM-5 and MiniMax M2.7 Catch Up to Claude Opus on Agent Tasks: $12 vs. $250 a Day
LangChain tested the open models GLM-5 and MiniMax M2.7 on basic agent tasks, including file handling, tool calling and instruction following, and found results on par with closed flagships. At a load of 10 million tokens a day, MiniMax M2.7 costs about $12, while Claude Opus 4.6 costs about $250, roughly $87 thousand a year. For high-volume, repetitive operations, open models are already worth trying, while the expensive model can be kept for planning.
AI-processed from LangChain Blog; edited by Hamidun News
Imagine an agent that all day moves files, calls APIs and parses instructions. If every step runs on the most expensive closed model, the bill grows very fast, and the user waits for a reply. The LangChain team tested whether such luxury is necessary.
According to its tests, the open models GLM-5 and MiniMax M2.7 (their weights can be downloaded and run anywhere) show results close to closed flagships on basic agent tasks: file operations, tool calling and instruction execution. At the same time, an application that processes 10 million tokens a day (a token is roughly a piece of a word) spends about $12 per day on MiniMax M2.
7 and about $250 on Claude Opus 4.6 from Anthropic. The difference comes to roughly $87 thousand a year.
What exactly was tested
Over recent weeks, LangChain ran open models through the tests of its Deep Agents framework. They checked what a model must have before it can be allowed into an agent harness at all: whether it can reliably call tools, follow structured instructions and work with files. There are seven test categories: file operations, tool use, information retrieval, dialogue, memory, summarization and "unit tests."
Each test has strict success conditions: the model either passes them or it doesn't. There are also soft efficiency checks, meaning how many steps and tool calls the model needed compared with what was expected. There are four metrics: the share of solved tasks (a score of 0.
68 means 68% of tests solved), a "correct and fast" indicator (expected number of steps divided by execution time; failed tests score zero), the ratio of actual steps to expected steps, and the ratio of actual tool calls to expected tool calls. A value of 1.0 means the model stayed exactly within budget, higher means it spent more, lower means it finished in fewer steps.
A model that solves a task in two steps instead of five wins on both accuracy and efficiency.
Reasoning settings were left at each provider's defaults. For Claude, that means the mode without extended thinking. LangChain itself stresses that the results are preliminary and the test suite will be expanded.
Money and speed
By the authors' estimate, closed flagships can cost 8–10 times more in high-load scenarios. The example of $250 versus $12 a day shows what this looks like in a budget.
The speed picture is similar. According to the reported data, GLM-5 on Baseten infrastructure responds in 0.65 seconds on average and produces 70 tokens per second. Claude Opus 4.6 takes 2.56 seconds and produces 34 tokens per second. Open models are usually more compact, and specialized inference providers such as Baseten, Fireworks or Groq optimize latency and throughput better than most teams could on their own. For interactive products, where a person watches the screen and waits, a gap like this is hard to close with engineering tricks.
How to try it
In the Deep Agents SDK, switching models takes one line in create_deep_agent: you install the provider package, for example langchain-baseten or langchain-openrouter, and specify the model name. The framework itself determines the size of the context window, disables unsupported modalities and adds the model name, provider and its limits to the system prompt. Context compression and summarization thresholds adjust automatically: for a model with a 4K context, compression is more aggressive than for Opus with 1M.
The providers tested are Baseten, Fireworks, Groq, OpenRouter and Ollama (cloud). The same model is often available from several of them, so you can choose by price, latency or privacy requirements. For fully local runs, Ollama and vLLM are suitable. You can run the tests yourself via pytest, specifying a model group or a particular model. In CI, LangChain runs one test suite across 52 models.
The CLI deserves a separate mention. It is an open coding agent that the authors describe as an alternative to Claude Code. It includes ConfigurableModelMiddleware, which allows switching the model in the middle of a session. This means you can plan a task on an expensive closed model and execute it on a cheap open one.
Limitations and conclusion
This is a publication from the company that sells the LangSmith agent platform, and the tests were written for its own harness. They check basic skills rather than the most complex reasoning and long chains of decisions, so "on par with closed models" applies to simple agent operations, not to all tasks. Latency figures depend on the specific provider and load.
The practical conclusion is simple. If your agent runs many repetitive file and tool operations, run GLM-5 or MiniMax M2.7 on your own tests and compare costs. The scheme "expensive model plans, cheap open model executes" looks like the most reasonable compromise. LangChain promises to describe harness-tuning techniques for families of open models and to test configurations where a closed orchestrator manages open subagents.
- Nov 22, 2026Axiom Math's AI Writes 43.1 Million Lines of Lean Proof in 24 Days
- Nov 22, 2026Yandex Grant Funds an LLM Engine: 35B Model on RTX 2080 Ti Runs Three Times Faster Than llama.cpp
- Nov 22, 2026Analysts estimate: by 2028, gas will supply 87% of Anthropic's power and coal 11%
- Nov 22, 2026Two AI Judges, 15 LLMs: Judges Favor Their Own Makers by 0.04 Points Out of 1
Key Questions Answered
What exactly was tested?
Over recent weeks, LangChain ran open models through the tests of its Deep Agents framework. They checked what a model must have before it can be allowed into an agent harness at all: whether it can reliably call tools, follow structured instructions and work with files. There are seven test categories: file operations, tool use, information retrieval, dialogue, memory, summarization and "unit tests." Each test has strict success conditions: the model either passes them or it doesn't.
What are the key details on Money and speed?
By the authors' estimate, closed flagships can cost 8–10 times more in high-load scenarios. The example of $250 versus $12 a day shows what this looks like in a budget. The speed picture is similar. According to the reported data, GLM-5 on Baseten infrastructure responds in 0.65 seconds on average and produces 70 tokens per second. Claude Opus 4.6 takes 2.56 seconds and produces 34 tokens per second.
How to try it?
In the Deep Agents SDK, switching models takes one line in create_deep_agent: you install the provider package, for example langchain-baseten or langchain-openrouter, and specify the model name. The framework itself determines the size of the context window, disables unsupported modalities and adds the model name, provider and its limits to the system prompt. Context compression and summarization thresholds adjust automatically: for a model with a 4K context, compression is more aggressive than for Opus with 1M.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.