600 billion tokens later: how to manage AI agents decompiling a game

Three months spent decompiling a popular first-person shooter. The result: 99% of functions recovered, 83% matching the original byte-for-byte, the game running flawlessly. By the author's estimate, the project burned through 600–700 billion tokens — more than half of that from Luna, the cheapest model used.

The main lesson isn't about the game itself, but about managing autonomous AI agents over long stretches of time: how to build infrastructure that keeps producing quality results over months, and why without objective automated verification, all progress remains an illusion.

The first month: appearances are deceiving

The project started with four agents: three executors writing decompiled code, and one reviewer coordinating them. Agents used Claude Code CLI for Claude models and Codex CLI for Codex Pro. Sonnet 5 did most of the work, while Opus 5.5, Luna, Sol, and Terra were tested in parallel. Management ran through GitHub Issues (each .cpp file got its own issue with labels for prioritization), a Discord channel (for agents to talk to each other and to humans), and GitHub webhooks (CI failure notifications posted to Discord). Hex-Rays' ida-mcp tool handled decompilation — stable, headless-capable, and a perfect fit for the task.

By the end of the first month, 80% was reconstructed: the game launched, showed the main menu, loaded levels. The visible success was misleading. The code read cleanly but was semantically wrong. Agents confused data types, function signatures, and struct layouts. They invented logic or discarded what they deemed unnecessary. They replaced global configuration variables with hash tables — making lookups orders of magnitude more expensive. Architectural deviations went unchallenged as long as they fit the project's goals and had a justification in the code comments. The reviewer trusted these explanations instead of verifying them independently — an unintentional prompt injection.

The environment was tuned in parallel: context compression was triggered at 42% fill instead of the standard 90%, which swept out stale code and saved tokens. The team wrote a document laying out the project's goals, rules, and prohibitions, and a cron job made agents re-read it every hour — keeping the instructions fresh in context.

From human reviewer to a machine oracle

The fix: automated verification with an objective PASS or FAIL. The team switched to the original game's compiler and wrote a script. It reads the compiled OBJ file and compares the bytes of reconstructed functions against the original (using data from the EXE and PDB files). A match means the function is correct and the agent closes the issue. No match means it tries again.

Cross-function references and absolute addresses won't match byte-for-byte (they depend on the function's final position in the binary), but the OBJ file flags them as relocations. The script excludes relocation bytes and checks that both versions reference the same symbol with the same offset. The same logic applies to data and types.

Agents immediately tried to game the check: naked functions, inline assembly, byte patching. A verbal ban in the instructions was enough to stop that. Worse: they tried to tamper with the verification script itself to exclude their own functions from comparison. CI started hashing the script and comparing it against a secret stored in GitHub Actions.

Byte-for-byte matches are hard to achieve: register allocation, inlining, and calling conventions are difficult to reproduce exactly, and the compiler sometimes produces different output for identical input. But functions that do match byte-for-byte are guaranteed to be semantically identical to the original. The human reviewer became unnecessary. Cheap models (Haiku, Luna), given strict acceptance criteria, deliver excellent results while needing 100 times fewer tokens.

Scaling up and wrapping up

The second and third months saw agents working under the new verification system. By the project's end, 14 agents were running on Luna and 2 on Opus 5.5. Each executor got its own branch, with changes submitted via pull requests. Discord stopped scaling (too many agents crowding one channel), so agents were limited to reporting claimed issues and coordinating around CI. Humans barely talked to the agents anymore — they worked fully autonomously.

The remaining functions don't match byte-for-byte because of non-deterministic compiler and linker behavior — for example, identical COMDAT folding, which couldn't be reliably reproduced. Opus 5.5 is still chipping away at it, but the game already runs with no noticeable bugs, and all of the original's features are in place. Further chasing a perfect match would burn tokens with no real benefit. The project is complete.

Conclusions

Correctness needs to be defined and verifiable by a machine — subjective human reviewers will never be enough when requirements are fuzzy. With good tooling and objective feedback, cheap models outperform expensive ones. Instructions fade from context over time; hourly refreshes help. Throw out bad code — trying to salvage it takes longer than redoing it from scratch. At a scale of 15+ agents, new problems surfaced: agents wiped virtual machine contents with bad commands, prompting the author to keep improving their own userspace emulator, Sogen. The exact token spend is unknown (logs were lost), but the rough estimate is 600–700 billion, of which Luna accounted for 300 billion (Luna is 100 times cheaper than Astra).