AI Agents Using GitHub's Off-the-Shelf MCP Server Cost 6.5× More Than CLI
An agent working with GitHub through an off-the-shelf MCP server (a set of ready-made tool "buttons") costs 6.51× more than one using a command-line interface (CLI) with a single flexible GraphQL query. The cause is not the protocol: each "button" returns the whole object, and that data piles up in the conversation history. With identical operations in a browser-based test, the difference is only 1.13×. Choose an interface based on the set of operations and the size of the responses.
AI-processed from Habr AI; edited by Hamidun News
If you connect an AI agent to your API, sooner or later you face a choice: give it an MCP server (a standard protocol that hands the agent a set of ready-made tools, such as "create a task" or "show comments") or a plain command-line interface, CLI. The developer behind the Takt service decided not to argue about it and to measure instead. In GitHub tests with the Sonnet model, the off-the-shelf MCP server cost 6.
51× more than a CLI with a single flexible GraphQL query, and it made 6.2× more calls. But the conclusion was unexpected: the protocol is not to blame.
What matters is which operations the agent has access to and how much data each call returns.
How it started
On June 8, 2026, an agent tried to detach a task from its parent on a Takt board. It sent parent="", but the MCP server received empty arguments. The cause turned out to be on the client side: it dropped the entire arguments object whenever at least one value was an empty string. This was already the fourth failure in two months of working with agents.
The author admits fault for two of the four problems: conflicting tool names (create_wiki and wiki_create) and about 40 tools that returned JSON encoded inside JSON. A third issue had to be worked around on the server side because of a client and framework he did not control. The empty-string problem could not be worked around at all, since the arguments never reached his code. That same day, he added a CLI to Takt, where the entire path from agent to HTTP API is under his control.
What the experiment showed
The author initially assumed the CLI won because of its architecture: MCP has to be compact, or it floods the context, but the more compact it is, the less it can do. To check this, he compared four ways of accessing GitHub and Takt: predefined CLI commands, a single GraphQL query through the CLI, an off-the-shelf MCP server, and a single graphql tool inside MCP. The agents were given 35 read-only questions with pre-verified answers. He tested Sonnet 5.5 in Claude Code and GPT-6.1 Sol in Codex: 280 runs plus 56 repeats.
At first, it looked like a rout: the off-the-shelf MCP servers required 3–6× more calls. But two parts of the theory did not hold up. MCP was not slower: its run time was 0.71–1.06× the CLI's. And the access method had almost no effect on answer accuracy: in most configurations, agents either answered everything correctly or got one question wrong. There were two exceptions: the predefined gh commands with Sonnet produced 17 correct answers out of 23, and Takt's off-the-shelf MCP produced 9 out of 12. The answers were checked against a reference key, not against the agents' own reports, because agents declare victory too early.
The author noticed a flaw in his design: he had changed two things at once, the protocol and the set of operations. So he ran a control test on his Browserious service, where the CLI and MCP can do the same things. On 27 paired tasks with Sonnet, MCP cost only 1.13× more, made 1.09× more calls, but finished in 0.85× the CLI's time. The author's caveat: these are different services and tasks, so one ratio cannot be carried over to the other, and a single experiment with one model does not prove equivalence in every case.
Where the money goes
A long list of tools turned out not to be the main cause. A short list is an architectural decision, not a protocol requirement: the gcloud MCP server has just one tool, while GitHub's has 91, and that one tool covers 15 of the 18 API families he studied. Moreover, Claude Code now loads only tool names at startup and fetches schemas as needed.
The main costs come from tool responses. Picture asking for a list of tasks and the server sending every task with all of its fields. That response lands in the conversation history and gets resent with every subsequent request. In the GitHub tests with Sonnet, uncached input tokens grew from 14,138 for the GraphQL CLI to 110,041 for the off-the-shelf MCP, and cache-write tokens grew from 13,193 to 109,084. That is roughly eightfold. A single request that names only the fields you need is cheaper than a dozen calls that return everything.
With GPT-6.1 Sol the picture is more curious: the off-the-shelf MCP returned an average of 789 KB per run, and in one case 18 MB, yet token usage rose only 1.33×. The author suspects Codex truncated large responses or moved them outside the context. This is a guess, not a measurement, but if it is right, the runtime environment is quietly deciding what the agent actually reads.
Secrets and the practical takeaway
The interaction method affected how passwords were handled. In the MCP configuration, the demo account's password had to be passed as a direct argument to page_fill, the tool echoed it back in its response, and it ended up in the model's context and logs. Agents using the CLI typed "$PW", and the actual value never appeared in the logs. The author stresses that this is a quirk of his configuration, not a limitation of MCP. He recommends passing references to secrets and substituting values on the server side, or authenticating before the agent starts.
The practical takeaway: when choosing an interface for an agent, look at the set of operations and the size of the responses, not the name. Let the agent request only the fields it needs, check answers against a reference key, and make sure passwords never enter the model's context.
- Nov 22, 2026Anthropic mathematician says Claude proved the Riemann hypothesis, but published only unreadable Lean code
- Nov 22, 2026Nasdaq 100 Falls 10.4% in a Week, Its Worst Stretch Since the Pandemic
- Nov 22, 2026Android work profile closes real-IP leak that Happ and v2rayNG don't catch
- Nov 22, 2026Anthropic Goes Public on Nasdaq: Shares Rise from $83 to $111, but Can Safety Hold Up?
Key Questions Answered
How it started?
On June 8, 2026, an agent tried to detach a task from its parent on a Takt board. It sent parent="", but the MCP server received empty arguments. The cause turned out to be on the client side: it dropped the entire arguments object whenever at least one value was an empty string. This was already the fourth failure in two months of working with agents.
What the experiment showed?
The author initially assumed the CLI won because of its architecture: MCP has to be compact, or it floods the context, but the more compact it is, the less it can do. To check this, he compared four ways of accessing GitHub and Takt: predefined CLI commands, a single GraphQL query through the CLI, an off-the-shelf MCP server, and a single graphql tool inside MCP. The agents were given 35 read-only questions with pre-verified answers.
Where the money goes?
A long list of tools turned out not to be the main cause. A short list is an architectural decision, not a protocol requirement: the gcloud MCP server has just one tool, while GitHub's has 91, and that one tool covers 15 of the 18 API families he studied. Moreover, Claude Code now loads only tool names at startup and fetches schemas as needed. The main costs come from tool responses. Picture asking for a list of tasks and the server sending every task with all of its fields.
Build & Ship Production AI Agents Without Boilerplate Code
Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.