Simon Willison→ original

Geoffrey Huntley fixes coding agents with a pre-commit hook and gets 10 out of 10

Geoffrey Huntley has described a way to make a coding agent use an internal diagnostics library. He considers updating AGENTS.md unreliable: a gap remains between the file and the agent's actual behavior, and it will keep growing. Instead, Huntley writes a pre-commit hook. The hook checks the git diff for direct calls to observability modules that bypass his diagnostics module. If it finds any, the commit fails and the agent receives a prompt telling it to use the library. By the author's own assessment, compliance rose to "ten out of ten." Simon Willison published the quote. • The rule checks the code, not the model's memory. • A rejected commit immediately becomes an instruction for the agent. • The author does not describe how "10/10" was measured.

AI-processed from Simon Willison; edited by Hamidun News
Geoffrey Huntley fixes coding agents with a pre-commit hook and gets 10 out of 10
Source: Simon Willison. Collage: Hamidun News.
◐ Listen to article

Geoffrey Huntley has described a technique that, by his account, brought an agent's compliance with an internal rule to "ten out of ten." He did not try to coax the model with yet another line in its instructions. Instead, he put a pre-commit hook in the commit path. Simon Willison published the quote, taken from Huntley's essay "nobody should type code by hand anymore. here's what's left." The metric is subjective, and the author does not say how it was measured. But the mechanics are worth a closer look.

Where AGENTS.md breaks down

Anyone who has worked with coding agents knows the situation. A project has its own diagnostics library, a wrapper around observability. The agent ignores it and calls observability modules directly. The obvious fix is to add a rule to AGENTS.md and hope the model follows it.

Huntley explains why that isn't enough. You can get the agent to analyze its own behavior and update the file, but a gap (a "delta") still remains between the instruction and actual practice, and it will drift. Text rules are probabilistic: the context grows, wordings compete with each other, the model or its version changes. A file of instructions remains a wish.

How the hook works

Huntley moves the rule from text into code. The setup is short:

  • Trigger: an attempt to run git commit.
  • Check: a script reads the git diff and looks for direct calls to observability modules that bypass the diagnostics module.
  • Reaction: if it finds a match, the commit fails.
  • Feedback: a prompt is injected into the agent telling it to use the diagnostics library.
  • Result, by the author's assessment: ten out of ten.

The last link in the chain matters. Given a bare commit rejection with no explanation, the agent could work around it in some random way. Here, the error message says both what is wrong and what to do about it. The agent gets the fix at the moment it needs it, not at the start of a session among hundreds of lines of context.

The check is also deterministic. It does not depend on what the model "remembers" from AGENTS.md, and it works the same on any model and in any agent tool that makes commits through git.

How to apply this yourself

The takeaway from Huntley's approach is an engineering one. Rules for agents fall into two groups: those that can be checked mechanically and those that cannot. The first group belongs in hooks, linters, and CI. The second, such as architectural taste or domain nuances, stays in AGENTS.md.

The candidates for migration are easy to spot: a ban on direct calls that bypass an internal wrapper, a mandatory logger, a single way of handling configs, forbidden dependencies. If a violation can be found with a regular expression or by parsing the diff, it should not live in a text instruction.

A few caveats to keep in mind (these are my own thoughts, not part of the quote):

  • Write the error text as a prompt: what was found, why it is not allowed, what to use instead.
  • A local hook can be bypassed with the --no-verify flag, so it makes sense to duplicate the same check in CI.
  • Watch for false positives: if the hook blocks legitimate commits, the agent will start "fixing" the code around the check rather than addressing the substance.
  • Keep a counter of hook triggers. It will show which rules the agent breaks most often, and it will also give you the metric that the "10/10" assessment lacks.

The tags on Willison's post, agentic-engineering and coding-agents, describe the direction accurately. Engineering around the agent, meaning hooks, checks, and feedback, is starting to matter as much as the choice of model.

ZK
Hamidun News
AI news without noise. Daily editorial selection from 50+ sources. A product by Zhemal Khamidun, Head of AI at Alpina Digital.
✦ Autonomous AI Agents

Delegate Deep Research, Coding & Workflows to an AI Agent

Deploy autonomous multi-step execution with NomadNet Agent (agent.nomadnet.ai) — automated web research, data synthesis, code generation, and complex task orchestration.

Launch NomadNet Agent →NomadNet AI Platform
What do you think?
Loading comments…