Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarHabr AI→ original

How to Keep an AI Agent Within Its Task: Code Checks the Plan Before Any Site Edits

You ask an AI agent to fix one page on a MODX-based site, and the agent decides to change a shared block (a chunk) instead. Several dozen pages that use that block change along with the page you asked about. The decision is technically sound, but nobody authorized it. The author moved control out of the prompt and into code: the ChangeSet component turns the agent's ideas into a plan and validates it before anything is written, without changing the site itself. Scope limits are therefore better enforced with programmatic checks than with a request in the instructions.

AI-processed from Habr AI; edited by Hamidun News
How to Keep an AI Agent Within Its Task: Code Checks the Plan Before Any Site Edits
Source: Habr AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHabr AI ↗
Editor-in-Chief
Key metric #1Two writes succeeded without errors, yet the first edit was lost (a lost update).
Key metric #2Before saving, the system compares the entire document, not just the revision number.
The approve() method rejects callsthe source="user" field cannot be trusted.

Imagine asking an AI agent to edit the text on one page of a website. The agent studies how the CMS is built and decides it is simpler to change a shared block (called a chunk in MODX). Technically, that decision is correct, but several dozen pages that include the block would change along with the page you asked about.

Nobody gave permission for that. A developer who first connected an agent to MODX via MCP (a protocol that gives a model access to external tools), and then set it up to work on live sites on its own, described how the Site Agent Runtime project addresses this problem. The idea is simple: the agent proposes, but the check is performed by ordinary code, not by the agent itself or by a prompt.

The key component is called ChangeSet. It prepares, validates, and commits a change plan, but it does not change anything on the site.

The full system is closed source, but the author has published three components (Task Manager, ChangeSet, and State Guard) on GitHub with automated tests. There is also an interactive demo: real Python code runs directly in the browser via Pyodide, so you can see which operations the system allows and which it rejects.

An Agent's Decision Is Not a Reason to Change the Site

The author separates three steps: decision-making, plan preparation, and execution. TaskSpec describes the task and the permitted actions. The Resolver maps them to specific site objects. DecisionSet records the agent's technical decisions. ChangeSet turns all of this into a verifiable plan, which is then picked up by a separate Executor.

The bind_scope() method binds each operation to a target found by the Resolver and checks the object type, its status, and the permitted action. If you allowed updating a resource and the agent tries to delete it, the operation is rejected. The same happens if it tries to touch an unrelated object. If an edit affects an element used by several resources, explicit confirmation is required, regardless of the risk assessment the agent itself reports.

Two Successful Writes and One Lost Change

Documents are stored in JSON files: with a modest number of tasks, a separate database is unnecessary. Writes go through a temporary file with flush, fsync, and os.replace(), and synchronization relies on an fcntl.flock lock. This looks reliable. But if two processes read the same revision, prepare different edits, and save them one after another, both writes pass without errors and the first result disappears. This is a classic lost update. Atomic replacement protects against a half-written file, but not against writing over stale data.

The author chose not to hold an exclusive lock for the whole time a plan is being prepared: one process would then slow down the others even when they do not need the storage. He chose optimistic concurrency control instead. The plan is prepared without a lock, and before saving, the document is reread and compared with what was read at the start. If they don't match, the system returns a "reload and retry" error instead of silently overwriting.

The entire document had to be compared, not just the revision number. The seal() operation moves a plan from draft to sealed, but it does not increment the revision number. A process holding an old copy of the draft could overwrite a plan that had already been sealed.

For race conditions like these, an ordinary test is almost useless: it can pass dozens of times while the bug is still there. In test_competing_writers_cannot_lose_revision, two threads read the same version and wait on a threading.Barrier. Then both try to save, and the test requires that exactly one succeed and the other receive a conflict. These are threads within a single process, but they exercise the real ChangeSet methods. A separate test simulates the case where the document is committed between the read and the write. The checks run in GitHub Actions on Linux with Python 3.10–3.13.

Integrity Is Not the Same as Authority

After seal(), a plain set() cannot change the content, otherwise the system would validate one plan and execute another. Changes are made through amend(): the method preserves history, increments the revision, returns the document to draft, and clears any previous approval. If the user approved a change to three resources, a fourth resource does not inherit that permission.

The plan's integrity is controlled by a canonical SHA-256 hash. Before sealing, it is checked against the draft's content, and for a sealed document it is recalculated during later checks. The test test_frozen_payload_recalculated alters the content of such a document and confirms that the plan is no longer considered ready for execution.

But a hash shows that a document was changed, not who changed it. If an agent calls approve(source="user"), that is just a string the agent can supply itself: it would be proposing a risky operation and also declaring that the user had approved it. That is why the published version of approve() rejects all calls, and plans that require confirmation cannot be sealed around it. There is no independent, trusted confirmation channel here, and the author does not pretend one exists.

A Correct Plan Can Still Go Stale

Even if a plan matches the task, is sealed, and passes the hash check, the site itself may have changed while the plan was being prepared. The agent read a resource, prepared an edit, and committed the ChangeSet, and in the meantime someone changed the object. To handle this, the plan stores a read_set: information about the source objects and their checksums, which is later used to verify that the data is still current.

The practical takeaway for anyone giving agents access to production: task boundaries, state versions, and confirmations must be enforced by deterministic code, not by requests in a prompt. Some of these problems, such as a lost write, have nothing to do with AI at all and are solved by ordinary engineering discipline. A caveat: the published components are demonstrations, not a ready-made library for managing websites.

Story

Key Questions Answered

What are the key details on An Agent's Decision Is Not a Reason to Change the Site?

The author separates three steps: decision-making, plan preparation, and execution. TaskSpec describes the task and the permitted actions. The Resolver maps them to specific site objects. DecisionSet records the agent's technical decisions. ChangeSet turns all of this into a verifiable plan, which is then picked up by a separate Executor. The bind_scope() method binds each operation to a target found by the Resolver and checks the object type, its status, and the permitted action.

What are the key details on Two Successful Writes and One Lost Change?

Documents are stored in JSON files: with a modest number of tasks, a separate database is unnecessary. Writes go through a temporary file with flush, fsync, and os.replace(), and synchronization relies on an fcntl.flock lock. This looks reliable. But if two processes read the same revision, prepare different edits, and save them one after another, both writes pass without errors and the first result disappears. This is a classic lost update.

What are the key details on Integrity Is Not the Same as Authority?

After seal(), a plain set() cannot change the content, otherwise the system would validate one plan and execute another. Changes are made through amend(): the method preserves history, increments the revision, returns the document to draft, and clears any previous approval. If the user approved a change to three resources, a fourth resource does not inherit that permission. The plan's integrity is controlled by a canonical SHA-256 hash.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…