Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarHabr AI→ original

Team Rules Changed a Coding Agent's Plan by 5 Points out of 5; Repeat Audits Changed It by 1

Excess context dilutes a coding agent's attention. The author built a Rust service with an agent, supplied data in batches, and rated after each one how much the plan changed: 1 meant nothing, 5 meant new sections and forks that require a human. Team rules and the team's repositories scored 5 out of 5: the agent learned about the internal framework redup-servicekit and switched its stack. Repeat audits scored 1.

AI-processed from Habr AI; edited by Hamidun News
Team Rules Changed a Coding Agent's Plan by 5 Points out of 5; Repeat Audits Changed It by 1
Source: Habr AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHabr AI ↗
Editor-in-Chief
2 of 24 steps got the maximum 5 pointsteam rules and example repositories
5 of 24 steps got 1 pointclarifications and three control audits with no new fixes
An early audit found a bugsync mode appended an extra /search to the address and returned a 404

When you give a coding agent a task, you want to attach everything: the spec, API documentation, team rules, code samples, "just in case." But the context window is finite, and surplus text dilutes the model's attention. The author of an article on Habr tested which context actually helps.

After each batch of data, the agent revisited its implementation plan and rated on a scale of 1 to 5 how much the plan had changed. A score of 1 means "the plan doesn't need to change," while 5 means "the plan has new sections and forks the agent cannot choose between, so a human is needed." For example, team rules scored 5 out of 5, while repeat code audits scored 1.

The practical takeaway: put into context what the agent cannot infer on its own. Repetitions and confirmations only burn tokens.

How the value of context was measured

The task was a real one: write a text search service in Rust on top of the Yandex Search API. The agent received context in batches, not in one piece. The author scored the plan, not the final code, because the plan reacts to new data faster than code does and shows well what the agent has actually taken into account.

The project folder held four files: readme.md with the project description, plan.md with the plan itself, logqa.md with the raw history of questions and answers, and influence.md with the scores for each batch. The first log answers the question "what happened," the second "how significant was it."

The scale looked like this:

  • 1 — the plan doesn't need to change;
  • 2 — cosmetic edits and renaming;
  • 3 — minor changes without new sections;
  • 4 — major changes and new sections;
  • 5 — new sections plus alternative paths the agent cannot choose between.

A 5 here means a human is unavoidable: the issue requires team agreements, security policies, or external constraints.

What upended the plan and what changed nothing

Of the 24 steps, two got the maximum. The first was the company's development rules. After them, the plan gained sections on observability, deployment, the repository model, and tests. Questions the agent could not answer also surfaced: REST/JSON or gRPC/proto, how to name repositories, whether helm and k8s are needed right away, how to ship logs to OpenSearch. These are team agreements, not technical matters.

The second was a set of the company's real repositories. The agent saw that the team already has an internal framework, redup-servicekit, with observability built in, and dropped the hand-assembled stack of metrics, prometheus, and tower-http. The config changed (YAML instead of toml), as did the way proto is connected (a git submodule) and the deployment scheme. None of this could have been guessed from an abstract spec: the knowledge lived in the team's heads and in its repositories.

External API documentation got 4 points: it turned out that Yandex Search API v2 was needed, not the old "Yandex.XML." A live request to the API later scored only 3: it clarified details but did not change direction.

Five steps got 1 point: a question about how to declare a dependency, a request to list the remaining decisions, and three control code audits in a row. After all the medium-severity findings had been fixed, the agent was asked several more times to "analyze and fix." Each time it answered that there was nothing new, no regressions, and the code had not changed.

That said, the audits in the first cycles paid off. One of them found a critical bug: sync mode appended /search to an already complete address and got a 404. The value of an audit dropped as the findings got fixed.

Practical takeaways

From the scale, the author draws several rules.

  • Put the spec in the project as a file. The agent can reread it at any moment, while a retelling in chat distorts the meaning and eats up the window.
  • Hand over team rules and real repositories first. They are what produce the 5s. If you supply them later, the plan has to be rewritten from scratch, which costs extra tokens and time.
  • Provide documentation for external systems as a source of facts. In parallel, keep a list of open questions and mark the blocking ones.
  • Hand 5-point forks to a human. If the agent hits a security policy or a team agreement, it should ask rather than guess.
  • Record closed decisions in the plan right away. Then the agent won't return to the question again.
  • Don't run repeat checks without new data. Each such run costs tokens and minutes, and the return is close to zero.

The author explains it this way: a high score means a space of choices has opened up before the human that did not exist before. A low score means everything is already determined. The usefulness of the same information falls as the plan takes shape: the first heavy batches give the most, and returns diminish after that.

Limitations

There is only one experiment: one task, one agent, one author. The agent itself assigned the scores using the given scale, so this is not an independent evaluation and not a benchmark. The results are unlikely to transfer one-to-one to other stacks and teams. But the method can be repeated at little cost: you just need to set up a log and a scores file and see which parts of your context actually work. You will also see where you are paying in tokens for noise.

Story

Key Questions Answered

How the value of context was measured?

The task was a real one: write a text search service in Rust on top of the Yandex Search API. The agent received context in batches, not in one piece. The author scored the plan, not the final code, because the plan reacts to new data faster than code does and shows well what the agent has actually taken into account.

What upended the plan and what changed nothing?

Of the 24 steps, two got the maximum. The first was the company's development rules. After them, the plan gained sections on observability, deployment, the repository model, and tests. Questions the agent could not answer also surfaced: REST/JSON or gRPC/proto, how to name repositories, whether helm and k8s are needed right away, how to ship logs to OpenSearch. These are team agreements, not technical matters. The second was a set of the company's real repositories.

What are the key details on Practical takeaways?

From the scale, the author draws several rules. Put the spec in the project as a file. The agent can reread it at any moment, while a retelling in chat distorts the meaning and eats up the window. Hand over team rules and real repositories first. They are what produce the 5s. If you supply them later, the plan has to be rewritten from scratch, which costs extra tokens and time. Provide documentation for external systems as a source of facts. In parallel, keep a list of open questions and mark the blocking ones.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…