A 30-Step Agent With a 1% Failure Rate Per Step Fails One Task in Four: How to Get It Into Production

In a demo, the agent behaves perfectly: 10 steps out of 10, everyone is happy. In production it runs thousands of tasks, and everything surfaces at once: the model confuses tools, the context is stuffed with junk, search fails to find what's needed, and the process crashes mid-task. The second part of the Habr series covers what to do about it. Here are the main points.

The math that makes a demo different from production

Imagine a task with 30 steps, where each step fails once in a hundred: the network blips, or the API replies "overloaded." The task then reaches the end 74% of the time. One in four fails. At a 3% failure rate per step, only 40% get through. The author is upfront that this is a simplified calculation: it assumes independent, transient failures. If a provider goes down entirely, three quick retries won't help. You need a long pause or a fallback provider.

Tools and context: fewer, but more precise

The model knows about a tool only its name, its description, its parameter schema, and what it returned. That is why you write a tool for an agent differently than an API for a programmer. Don't build one tool per endpoint: instead of list_users, list_events, and create_event, it is better to have a single schedule_event that finds a time and creates the meeting itself.

Keep responses short, with readable names instead of uuids. In an Anthropic example, a concise mode for a Slack tool returned 72 tokens instead of 206. Write errors for the model: a message like "the date must be in the format 2026-11-09, got 09.

11" will fix itself on the next turn, while a bare 400 code tells the model nothing.

The same principle applies to context: keep the smallest set of tokens that helps solve the task. Load data as you go, leaving only references in the prompt. Compress the history, preserving decisions and open bugs. Hand heavy chunks of work to subagents, which return a summary of 1–2 thousand tokens.

RAG: first decide whether you need it

If the knowledge base is smaller than 200 thousand tokens (about 500 pages), you can put all of it in the prompt and cache it. No search is needed then. If the base is larger, Anthropic's measurements show that a combination helps: semantic search plus keyword search (BM25), 50–100 tokens of context added to each chunk before indexing, and reranking. The failure rate, where the needed chunk did not land in the top 20, fell from 5.7% to 1.9%. Test search separately from the answer: build questions with a known answer location and count how often the chunk appears in the results. Otherwise it is unclear whether search or the model is to blame.

A loop under control: turns, money, retries

Start with a chain, not an agent: an agent pays for its freedom with latency and money. If you do need one, set three limits: a turn limit, a dollar budget, and a timeout for the whole task. Cost grows faster than the number of turns, because every turn re-reads the history. By the author's calculation, a 30-turn task costs $0.51, while a stuck agent at 100 turns costs $2.68: 3.3 times more turns, 5.3 times more money. Decide in advance where the task goes when attempts run out: a fallback model, a simple chain, or a human.

Retry only transient failures such as 429, 500, 504, and 529, with growing pauses. The catch is that a 429 can also mean the monthly spending limit is exhausted (error code enforced_spend_limit_reached). A retry is then useless, and a retry policy that doesn't tell the two apart will hammer the API for hours.

The danger is not the crash but the repeat

Imagine a send_email tool: the email went out, but the response was lost, the wrapper repeated the call, and the customer received two emails. With charge_card, the money is likewise charged twice. The solution is known from Stripe: an idempotency key.

The server remembers the response to the first request with a given key and returns the stored response on a repeat. The agent already has such a key: the tool call id, which does not change on a retry. Pass it on to the external system.

But if the process crashes between sending and recording the result, a local check will not catch the duplicate. Send actions, payments, and deletions, as well as any actions the external system does not support keys for, should go to a human for confirmation.

If the process itself crashes, there is nothing to retry: the history and results were held in memory. This calls for durable execution, for example Temporal. The agent loop lives in a workflow, model and tool calls become activities, events are written to a journal, and another worker picks up from the point of failure. The condition: the workflow code is deterministic and the activities are idempotent. The price of skipping this: a crash on turn 25 with a restart from scratch costs $0.93 instead of $0.51, which is +81%.

What to do in practice

Cut the set of tools and make them coarser-grained. Set a turn limit, a budget, and a plan B. Check that every action with consequences is either idempotent or goes through confirmation. Measure the cost of a successful task, not of a single call, and run checks on every prompt change.

Limitations: all cost calculations are simplified and tied to the scenario from the first part of the series, and some of the figures on limits and errors depend on the documentation as of a specific date. The section on observability and security is cut off in the available text, but the general advice is clear: don't give an agent private data, untrusted content, and an outbound channel at the same time.