Agents behave differently over MCP and CLI, and the DART method added up to +9 points

When an AI agent uses tools, it does not see the "pure" capabilities. It sees a description of them: either a set of functions exposed via MCP (Model Context Protocol, a standard for connecting tools to a language model), or commands in a terminal, i.e.

a CLI. A team from ModalityDance (Zhenning Shi, Liang Pang, Yongqi Li) asked a direct question: if the capabilities are exactly the same and only the way they are shown to the model changes, will the agent's behavior change? The answer is yes, and substantially.

Task-solving quality, token consumption (the units of text you pay for when working with a model), and even the way the agent makes mistakes and recovers from them all diverged. The authors also proposed a method called DART, which improved results in all six tested configurations by up to 9.0 points.

How to make the comparison fair

MCP and CLI are usually compared "as is." That makes it unclear what actually drove the result: the interface itself, or the fact that one has richer tools or implements them differently. To remove that confusion, the authors built a testbed called MICA (Matched Interface Comparison and Analysis).

It works like this:

• there is a single contract of semantic actions: a list of everything the agent can do; • two versions are generated automatically from that contract: MCP and CLI; • whichever one the agent picks, the action is decoded and sent to the same executing code.

Under the hood, the capabilities match completely. The only difference is in the "storefront": how actions are described, how they have to be written, and what the environment's response looks like. The tests were run on six models in three interactive environments.

Where the agent's paths diverge

First result: performance and interaction cost differ noticeably. And it is not just a matter of syntax typos, where the agent wrote a call incorrectly. The interface affects which action the agent chooses in terms of meaning, how it fails, and how it recovers from a failure.

To find the source of this sensitivity, the researchers used two techniques.

• Same-state probes. The agent was placed in the same situation, and three things were changed one at a time: how the capabilities are presented, how the action is written, and how feedback is returned. This showed which of the three throws the model off the most. • Paired natural trajectories. The agent solved the same task through MCP and through CLI. Of 2004 episodes, a structural divergence appeared in 94.3%, or roughly 1890. Early differences did not fade out but carried forward: one bad step at the start changed the entire rest of the interaction.

The authors' conclusion: the interface is part of a closed decision loop. The agent acts, receives a response, and decides again, and the form of both the response and the action syntax influences every subsequent step. It is not a neutral wrapper around the model.

DART: fix the command syntax, not the decision

Since the divergence arises at the stage of "realizing" the action, that is where it can be corrected. DART (Decoupled Action Realization and Targeted Repair) splits the work into two parts. First, the agent determines what exactly it wants to do. This semantic decision is checked and saved. Then a separate step handles how to write that decision in the language of the specific interface, and when errors occur, it repairs only that written form. The meaning of the chosen action is not revisited.

The effect, according to the reported figures:

• benchmark scores rose in all six tested configurations (model plus environment), by up to 9.0 points; • MCP cost fell in all three environments, by up to 27.1K tokens per task.

The second figure matters for budgets. MCP tool descriptions and responses are often verbose, and they have to be pushed through the context at every step. If part of the work is not repeated, consumption drops noticeably.

What this means for engineers

The "MCP vs. CLI" debate loses its meaning when framed this way: you cannot determine a winner at all without controlling how the interface is implemented. The same model with the same tools can look stronger or weaker depending on how they are presented.

Practical takeaways:

• when comparing models and agent frameworks, fix the interface and describe it in the report, otherwise the numbers cannot be compared; • analyze agent failures by level: did it err in choosing the action, or only in writing the call; • if you build an agent on MCP and pay for tokens, it makes sense to separate the semantic decision from its formatting and fix only the formatting.

The limitations are also visible: the study covers six models and three environments, so the numbers should be carried over to any other set of tasks with caution. The code and project page are open (the ModalityDance/MICA repository on GitHub), so you can verify the results on your own agents.