Habr AI: Why Agentic Systems Need New Control and Safety Metrics

The shift from chatbots to agentic systems changes what has to be controlled. It used to be enough to judge how useful and correct a model's reply to a request was. Now the whole chain of actions the model builds for itself needs evaluation. An agent does not just generate text: it plans steps, picks tools, requests data, may delegate part of the work to other agents and makes intermediate decisions. In such an architecture, a polished final answer no longer guarantees that the system worked reliably, safely and economically.

For a classic chatbot, the main metrics were usually answer quality, accuracy of wording, relevance and user satisfaction. For an agent these are not enough, because an error can appear long before the final message. The system can split a task into stages incorrectly, choose the wrong tool, end the scenario too early, get stuck in repeated actions, or conversely take extra steps and spend too many tokens, too much time and too many external requests. So the focus falls not only on the result but also on the trajectory the agent followed to reach it.

For this reason the set of metrics expands. First comes the share of successfully completed tasks: not merely whether the agent gave a plausible answer, but whether it reached the user's goal without manual intervention. Next are planning-quality indicators: how logical the chosen steps are, how many of them are actually necessary, and how often the plan has to be rebuilt during execution. Tool-call correctness needs separate measurement: did the agent pick the right API, pass valid parameters, get the expected result and handle an error adequately? For multi-agent systems, coordination metrics are added: do agents duplicate each other's work, lose context or create conflicting actions?

Cost and observability matter just as much. Agentic systems are almost always more expensive than an ordinary dialogue, because every extra step and every call to a model or external service has a price. Control must therefore track the average number of iterations per task, token consumption, frequency of repeated requests, execution time and the share of pointless actions. Tracing requirements grow in parallel: the team needs to see what decision the agent made at each stage, what data it relied on, why it chose a particular tool and at what point it deviated from the expected scenario. Without this transparency, it is impossible to debug behavior, investigate failures or prove compliance with internal policies.

Safety requirements change as well. A chatbot mainly risked producing incorrect or dangerous text; an agent can take an undesirable action: send a request to the wrong place, gain excess access to data, modify a record in a system or use a tool outside its permitted context. An agentic architecture therefore needs granular access rights, a sandbox approach for tools, strict policies on executing actions, limits on autonomy and stop mechanisms if the system starts behaving suspiciously. Safety here stops being a filter on input and output and becomes part of the operational loop.

Another shift concerns operations. For an agentic system, it is not enough to complete a task in the ideal scenario; it must also degrade safely when something fails. Recovery metrics become useful: how often the agent can fix its own mistake, when it hands the task to a human, how many incidents require manual review and how quickly the team can reproduce a problem from logs. In practice, product and platform teams must design not only the agent's intelligence but also its failure modes, monitoring and intervention procedures.

The main takeaway is that agentic systems cannot be evaluated by the same rules as ordinary chat interfaces. Companies have to move from checking answer quality to full-fledged execution engineering: measuring task completion, plan robustness, correctness of tool calls, cost, traceability and compliance with safety rules. The more autonomous an LLM becomes, the closer controlling it comes to monitoring a complex software service, and the further it moves from editing good or bad replies.