Developer of an AI-Agent-Built Game: 30 Green Tests Proved Nothing Until He Wrote a New One

A familiar situation: you explain the same thing to an agent for the third time, and for the fourth time it does it its own way. The author of the strategy game Age of Poleis, where AI writes the code while he himself comes up with ideas, makes decisions and plays, described how he got out of this loop. His experience is useful to anyone who trusts agents with routine work.

Rule, Method and Check

The stack: a Django backend, and a React and TypeScript interface. The contract is described on the backend, an OpenAPI schema is produced from it, and TypeScript types are generated from the schema. The temptation is obvious: a frontend type doesn't match the API response, and you want to fix it by hand, since it's a single line. But on the next generation the fix quietly disappears, and the bug comes back.

When the author noticed he was explaining this not for the first time, the rule moved into the agent's permanent instructions: don't touch generated files by hand, change the source and regenerate, and regenerating again must change nothing. But text in an instruction remains a wish. So a step appeared in the shared check script: it generates the schema and types into temporary files and compares them with what is in the repository. If there is a discrepancy, the check goes red.

The formula is simple: a rule, a way to follow it, and a check. Without a check, the rule works as long as the agent remembers it. With a check, it works always. The author handles formatting, localization and other repetitive routine the same way.

Thirty Green Tests That Checked the Wrong Thing

In the Windows version, the game launched in three windows: loading, the game, and waiting on exit. The author wanted one window, with loading and the game running inside it. In words the task is a single line; in code it is several states: launch, loading, closing in the middle of launch, saving, restarting.

All 30 existing shell tests were green. But they checked the old behavior, which the author specifically wanted to move away from. Green meant "works as before", while what was needed was "works differently". So the first thing to appear was a new window lifecycle test. On the old code it failed, because it looked for the main window and found the loading window. After the rework it passed.

So that the agent can verify the result itself, the author phrases the desired outcome as observable facts: loading and the game in one window, closing during launch does not open a new window, a saved campaign is still there after a restart. In the end 96 JavaScript tests passed, and the agent separately ran the real Electron application.

Two more techniques. First: in every report the agent must write what was not verified. Here it was installation on a clean Windows. A month later, such a line protects against false confidence. Second: a screenshot does not prove the required state. The first frame showed the campaign preparation menu, not the game itself. Only a later frame from within the campaign, together with a check of the real window, served as confirmation.

What Remains for the Human

When the technical part is verified without the author, he plays with different questions in mind: not "does the button work" but "do I want to press it". Mechanics can be flawless and boring, and an economy easily turns into bookkeeping. No test will catch that. Assigning residents to professions was fun for the first few minutes, but after a hundred game periods it became tiring, and in the end automatic candidate selection by criteria appeared.

Perfectionism is a separate trap. The author replayed the start dozens of times and fixed crooked buttons, even though randomness in the game makes every match different anyway. He himself concedes that personal enjoyment doesn't replace other people's playthroughs.

Builds, signing, certification and marketing remain with the human. AI will draft a description in five minutes, but it won't decide who needs the game and what to show first.

Practical Takeaways

• Explained something twice? Turn it into a rule and immediately into an automated check.
• To change behavior, write a test that fails on the old code.
• Demand from the agent a list of what was not verified and proof of state, not just a screenshot.
• Hand the agent whatever can be measured. Keep taste and the feeling of "is it interesting" for yourself.