Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarLangChain Blog→ original

Open GLM-5 and MiniMax M2.7 Catch Up to Claude Opus on Agent Tasks: $12 vs. $250 a Day

LangChain tested the open models GLM-5 and MiniMax M2.7 on basic agent tasks, including file handling, tool calling and instruction following, and found results on par with closed flagships. At a load of 10 million tokens a day, MiniMax M2.7 costs about $12, while Claude Opus 4.6 costs about $250, roughly $87 thousand a year. For high-volume, repetitive operations, open models are already worth trying, while the expensive model can be kept for planning.

AI-processed from LangChain Blog; edited by Hamidun News
Open GLM-5 and MiniMax M2.7 Catch Up to Claude Opus on Agent Tasks: $12 vs. $250 a Day
Source: LangChain Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceLangChain Blog ↗
Editor-in-Chief
Key metric #1GLM-5 on Baseten responds in 0.65 s, Claude Opus 4.6 in 2.56 s
Generation speed70 tokens/s for GLM-5 versus 34 for Opus 4.6
Key metric #3Switching models in Deep Agents is one line of code, but the tests are early

Imagine an agent that all day moves files, calls APIs and parses instructions. If every step runs on the most expensive closed model, the bill grows very fast, and the user waits for a reply. The LangChain team tested whether such luxury is necessary.

According to its tests, the open models GLM-5 and MiniMax M2.7 (their weights can be downloaded and run anywhere) show results close to closed flagships on basic agent tasks: file operations, tool calling and instruction execution. At the same time, an application that processes 10 million tokens a day (a token is roughly a piece of a word) spends about $12 per day on MiniMax M2.

7 and about $250 on Claude Opus 4.6 from Anthropic. The difference comes to roughly $87 thousand a year.

What exactly was tested

Over recent weeks, LangChain ran open models through the tests of its Deep Agents framework. They checked what a model must have before it can be allowed into an agent harness at all: whether it can reliably call tools, follow structured instructions and work with files. There are seven test categories: file operations, tool use, information retrieval, dialogue, memory, summarization and "unit tests."

Each test has strict success conditions: the model either passes them or it doesn't. There are also soft efficiency checks, meaning how many steps and tool calls the model needed compared with what was expected. There are four metrics: the share of solved tasks (a score of 0.

68 means 68% of tests solved), a "correct and fast" indicator (expected number of steps divided by execution time; failed tests score zero), the ratio of actual steps to expected steps, and the ratio of actual tool calls to expected tool calls. A value of 1.0 means the model stayed exactly within budget, higher means it spent more, lower means it finished in fewer steps.

A model that solves a task in two steps instead of five wins on both accuracy and efficiency.

Reasoning settings were left at each provider's defaults. For Claude, that means the mode without extended thinking. LangChain itself stresses that the results are preliminary and the test suite will be expanded.

Money and speed

By the authors' estimate, closed flagships can cost 8–10 times more in high-load scenarios. The example of $250 versus $12 a day shows what this looks like in a budget.

The speed picture is similar. According to the reported data, GLM-5 on Baseten infrastructure responds in 0.65 seconds on average and produces 70 tokens per second. Claude Opus 4.6 takes 2.56 seconds and produces 34 tokens per second. Open models are usually more compact, and specialized inference providers such as Baseten, Fireworks or Groq optimize latency and throughput better than most teams could on their own. For interactive products, where a person watches the screen and waits, a gap like this is hard to close with engineering tricks.

How to try it

In the Deep Agents SDK, switching models takes one line in create_deep_agent: you install the provider package, for example langchain-baseten or langchain-openrouter, and specify the model name. The framework itself determines the size of the context window, disables unsupported modalities and adds the model name, provider and its limits to the system prompt. Context compression and summarization thresholds adjust automatically: for a model with a 4K context, compression is more aggressive than for Opus with 1M.

The providers tested are Baseten, Fireworks, Groq, OpenRouter and Ollama (cloud). The same model is often available from several of them, so you can choose by price, latency or privacy requirements. For fully local runs, Ollama and vLLM are suitable. You can run the tests yourself via pytest, specifying a model group or a particular model. In CI, LangChain runs one test suite across 52 models.

The CLI deserves a separate mention. It is an open coding agent that the authors describe as an alternative to Claude Code. It includes ConfigurableModelMiddleware, which allows switching the model in the middle of a session. This means you can plan a task on an expensive closed model and execute it on a cheap open one.

Limitations and conclusion

This is a publication from the company that sells the LangSmith agent platform, and the tests were written for its own harness. They check basic skills rather than the most complex reasoning and long chains of decisions, so "on par with closed models" applies to simple agent operations, not to all tasks. Latency figures depend on the specific provider and load.

The practical conclusion is simple. If your agent runs many repetitive file and tool operations, run GLM-5 or MiniMax M2.7 on your own tests and compare costs. The scheme "expensive model plans, cheap open model executes" looks like the most reasonable compromise. LangChain promises to describe harness-tuning techniques for families of open models and to test configurations where a closed orchestrator manages open subagents.

Story

Key Questions Answered

What exactly was tested?

Over recent weeks, LangChain ran open models through the tests of its Deep Agents framework. They checked what a model must have before it can be allowed into an agent harness at all: whether it can reliably call tools, follow structured instructions and work with files. There are seven test categories: file operations, tool use, information retrieval, dialogue, memory, summarization and "unit tests." Each test has strict success conditions: the model either passes them or it doesn't.

What are the key details on Money and speed?

By the authors' estimate, closed flagships can cost 8–10 times more in high-load scenarios. The example of $250 versus $12 a day shows what this looks like in a budget. The speed picture is similar. According to the reported data, GLM-5 on Baseten infrastructure responds in 0.65 seconds on average and produces 70 tokens per second. Claude Opus 4.6 takes 2.56 seconds and produces 34 tokens per second.

How to try it?

In the Deep Agents SDK, switching models takes one line in create_deep_agent: you install the provider package, for example langchain-baseten or langchain-openrouter, and specify the model name. The framework itself determines the size of the context window, disables unsupported modalities and adds the model name, provider and its limits to the system prompt. Context compression and summarization thresholds adjust automatically: for a model with a 4K context, compression is more aggressive than for Opus with 1M.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…