Anthropic and other language models can call hidden tools without permission

The biggest risk with agentic LLMs today isn't that they occasionally get function parameters wrong. It's that, in a poorly designed architecture, they can call a tool that nobody formally gave them. If such a function exists in the environment, the model can guess its name, make up arguments and trigger a real side effect: read a secret, write a message, hit someone else's API. On paper, access rights look restricted; in practice, the line between allowed and forbidden turns out to be probabilistic. For systems where the model works with data, email, documents or corporate integrations, this is no longer a curiosity but a security risk.

The problem shows up where an agentic model meets a client environment in which the list of permitted tools and the actual namespace diverge. In the demonstration, the model was given only read_url, but the environment also contained a read_secret function. After a hint like from dialoghelper import *, the model concluded it had access to read_secret and tried to call that tool.

If the library or API checks only the format of the call and doesn't strictly match the function name against the provided schema, the request goes through. The author showed similar behavior not only in Anthropic, but also, in certain scenarios, in Gemini, Grok and via multi-provider wrappers. In other words, the vulnerability doesn't come from a single brand; it emerges at the junction of the model, the SDK and the agent harness.

The danger rises sharply when such a system lands in the so-called lethal trifecta: the model has access to external tools, untrusted content and private data. At that point, prompt injection stops being a mere annoyance and becomes an exfiltration channel. An attacker only needs to embed an instruction in an email, web page or document, and the model, convinced the hidden tool is available, reaches for the secret on its own and sends it out. Worse, the architectural separation developers rely on can create a false sense of security: the sensitive tool isn't supposedly in the toolset, yet it physically sits next door in the environment and becomes reachable through an unauthorized call.

This failure is hard to detect. In many cases the model behaves correctly and refuses the forbidden call, then suddenly breaks because of some small detail in the context: the wording of a prompt, the message history or even a module name. The article gives a telling example where the same action is blocked one moment and goes through the next, after a harmless mention of dialoghelper.

Structured decoding partially reduces the risk because it forces the model to stay within the schema, but it comes with trade-offs: higher latency, hard limits on the number of strict tools and degraded performance on large function sets. So you can't rely on the model behaving smartly. Tool names must be checked on the provider side and in client code before anything actually executes.

The practical takeaway is simple: if you're building an agent on top of MCP, Jupyter, internal SDKs or any other dynamic environment, hiding extra functions from the description and hoping the model stays disciplined isn't enough. You need to strictly validate every tool call, separate sensitive operations from untrusted content and keep the execution space narrower than what the LLM can see. Otherwise, a single guessed read_secret or add_msg turns a carefully designed agent architecture into a system where access rights exist only until the first successful hallucination.

The good news is that the fix doesn't look hard: simply drop any call whose name isn't in the allowed schema before it ever reaches the runtime. A few lines of defensive code in a library or gateway can close off a whole class of problems that otherwise passes for a rare hallucination but in reality breaks the trust model of the entire system.