Habr Author Proposes Acceptance Testing for AI Agents: "Done" Requires Proof

The most expensive mistake a working AI agent makes looks like success: a short "done" with nothing behind it. The author of a recent Habr article proposes shifting the focus from the quality of the answer to the acceptance of the work: what exactly must happen before a task can be considered closed. The thesis in the introduction is blunt. An assistant that suggests phrasings can be forgiven an awkward paragraph. An agent that changes working data cannot.

The publicly available excerpt contains no benchmarks or specific methods; those remain in the full version on Habr. But the framing of the question is accurate and worth unpacking, because this is exactly where the line between a demo and production now runs.

Why "done" proves nothing

An assistant produces text, a person reads it and decides what to do with it. The checkpoint is built into the process. An agent acts on its own: it writes to a CRM, edits spreadsheets, runs scripts, sends emails. Its report that "the task is complete" is still generated text, like any other model output. It may match the state of the system, or it may not.

The rest of the scenario is familiar. A colleague goes to check, finds discrepancies, redoes the work by hand and explains themselves to the people affected. The cost of the error shifts from the agent to humans, and it does so invisibly: in the automation statistics, the task is still listed as completed.

Acceptance instead of trust

The logic of acceptance is borrowed from ordinary engineering. Criteria for closing a task are fixed before launch and checked against the state of the system, not against the agent's retelling. The record has actually been created. The required field has been changed. The number of affected rows matches the expected count. Nothing extra has been touched.

Engineering practice usually distinguishes three levels of such verification:

• result: the target state has been reached, and this is visible in the system itself;
• boundaries: a before-and-after comparison of the state shows only the planned changes;
• trace: the agent's action log makes it possible to reconstruct what exactly it did and in what order.

The levels complement one another. A result without a boundary check will miss side edits, and a log alone, without a result check, turns into a nice story about how well everything would have gone.

Which metrics to track

Until acceptance is measured, the discussion stays at the level of impressions. Here is a minimal set of indicators for a team that puts an agent into a working process:

• share of false "done" reports: tasks declared closed that did not pass verification;
• share of tasks accepted without human edits;
• human time spent verifying a single task;
• share of tasks for which verification is automated;
• number of change rollbacks per period.

It is important to look at these figures together. If an agent closes many tasks but the time spent on manual rechecking grows, the savings are fictitious.

Where to start in practice

Pick one task the agent already performs, and before the next run, write down its closing criteria in a verifiable form. If a criterion cannot be checked with a script or a query, it is too vague and will have to be reformulated.

Next, separate the verifier from the executor. A check that the same agent runs and interprets itself repeats the original problem. A simple database query or a comparison of exports from before and after is more reliable than a self-report.

Finally, leave humans in the role of spot-checkers where an error is costly: bulk data changes, payment operations, external mailings. For everything else, automatic acceptance is enough, with a clear threshold above which the task goes back for manual review.

The Habr author's idea boils down to a simple rule: an agent gets the right to say "done" only together with proof. For a market where agents are being rolled out en masse into working processes, this is a sound and inexpensive standard.