Makers of an AI manager for marketplaces deleted the agent and cut 461 lines of code

Four days after launch, the team building an AI manager for marketplace sellers deleted its agent entirely. Internal tests made it look like a success, but live conversations told a different story: the answers sounded confident while drawing on only a small share of the data. Instead of patching it, the developers threw out their own core and moved the agent's run loop to Pydantic AI. The commit was telling: 582 lines removed, 121 added.

The tests said one thing, the conversations another

For evaluation, the team built a benchmark: eight typical tasks for each role, plus a judge model that scores each answer from 1 to 5 and sees the data the agent answered from. On this benchmark the agent beat the old phrase parsing for seven of nine roles, and the marketplace manager scored 4.28.

Then the authors read the actual conversations. An owner asks why sales dropped, and the agent looks at ten products out of a hundred: the tool returns "up to 15 results," and the table passed to the model is truncated further. The conclusion is smooth, but it rests on a tenth of the store. The judge does not catch this kind of error, because the data the agent saw is consistent with its conclusion. It is a typical blind spot of LLM-based evaluation: the judge checks whether an answer is consistent with its context, not whether the context itself is complete.

The other problems turned out to have the same origin:

• The reviews wrapper returned recent reviews, not all of them. A request like "go through everything for the last six months" ran into a missing method, and the agent answered "can't do that yet." The list of such requests in the admin panel grew faster than the team could write new wrappers.
• Tool descriptions contained instructions like "one or two calls with broad terms" and "up to 15." The model followed them even where the task required twenty calls.
• A limit of eight steps per run was not enough for tasks like "figure out why a niche is declining." The team raised it to 50 steps and 15 minutes, but the effect was small: the agent was not hitting the step limit, it was getting only a sliver of data at each step.

The authors' conclusion: they were afraid of giving the agent freedom, and decided for it how much to look at, what to show, and how to use a tool. Each decision looked reasonable, but together they turned the agent into an executor that does exactly what the instructions say.

Why they threw out the core instead of fixing it

The in-house core included the loop, tool calls, retries after dropped connections, compression of old steps, and tool groups. All of it needed maintenance, and none of it was unique: readers of the previous article rightly pointed out that it is plain ReAct. The product's value lies elsewhere: in the tools for the marketplaces, in the rules for writing to the seller account, and in how the answer is presented to the owner.

So the team replaced the loop with Pydantic AI. Rarely used tools are loaded through DeferredLoadingToolset, only via search, rather than sitting in the context of every step. From the ready-made set, the team took DuckDuckGo search with eight results and page loading by link. The request limit is switched off: usage_limits.request_limit is None. The team's own tools stayed the same and are called through the same functions, so the tables shown to the owner, the action log, and the write rules did not change.

One tool instead of dozens of wrappers

The main change is one of principle. Before, the agent was given tools and told how to use them. Now it gets access, a tool, and a reference, and chooses how to use them itself.

The entire Wildberries API became a single tool that exposes 307 methods, plus a separate tool with the official reference. Product cards, prices, stock, orders, supplies, advertising, reviews, chats, and finance are available to the agent to the same extent as to the API itself. If a method is not in the reference, the tool returns a hint to check it with a reference call. Read methods run immediately and in full, without truncation. The API of the MPStats analytics service is connected the same way. The knowledge library was removed from the prompt: the reference sits next to the API, and the agent searches the web for everything else on its own.

One rule survived the rewrite: any write to the seller account happens only after the owner says "yes." The mechanics changed. Writes made during a run accumulate into a single batch, for example "change the price on three products, raise the bid in two campaigns, reply to four reviews." The owner sees the list in a "was → will be" format and answers once. The batch runs in order and stops at the first error.

Among the smaller improvements: parallel reads (while the agent looks at orders, it is already checking stock and advertising), edits made strictly one at a time, streamed text output with statuses like "reading March reviews," remembering successful API calls within the project, and resuming a run from the same point after a model or search failure.

The authors have not yet published results for the new version. They only write that the agent now digs deeper, and that this made them wonder whether it is using the right model. The article gives no figures on quality or on token cost with the limits removed. For teams building agents on top of closed APIs, the takeaway is practical: first check whether your own wrappers are truncating what the model should see in full.