An Instruction Is Not a Guarantee

A little while back I left a comment on a post by Dr. Maryam Miradi about spec-driven AI engineering — the practice of writing down, in real detail, what an agent is supposed to build before it writes a single line of code. I meant every word of that comment. Writing the spec first stopped being optional a while ago. It's the difference between directing an engineer and hoping a competent one shows up in the diff.

But somewhere over the last year, Claude Code stopped being something I reviewed line by line and became something that ran for hours at a stretch — reading a spec, breaking it into tasks, writing the code, running its own tests, deciding for itself whether to move on to the next step. Once that happened, I noticed the spec was answering the wrong question for the failure modes I was actually seeing. A spec is very good at telling a model what to build. It has almost nothing to say about how the model should behave while it's building it — what it's allowed to skip, what "done" is allowed to mean, whether it gets to grade its own homework.

So I wrote instructions for that too. Never commit straight to the main branch. Never mark a test as passing without actually running it. Never skip a step because it looks trivial this time. All reasonable rules, all correct — and all, in the end, just sentences in a markdown file. A model reads them, agrees with them, and then, deep into a long autonomous run, quietly stops acting like it ever read them. Not out of defiance. Attention isn't memory, and a rule that isn't in front of the model right now has to compete for space with everything that is.

That's the part I think gets missed in most of the spec-driven conversation. A better-written spec does not fix an attention problem. You can phrase "never do X" as clearly as language allows, and it is still a sentence the model has to remember to apply, every single time, for the rest of the session. If I really mean never, wanting isn't a mechanism 😄. So at some point I stopped trying to word the instruction better and started building the thing that would enforce it whether the model remembered or not.

In practice that turned into four pieces of process wrapped around the model — not inside its prompt, around its work.

Put together, that's a small closed loop, and it's worth drawing out properly, because the loop is the actual point:

What this looks like on disk

None of this needed custom infrastructure. It's built entirely out of Claude Code's own extension points — two of them apply to your whole machine, the rest are just habits you wire into them.

---
name: database-migration
description: How schema changes get made safely
---

Before writing a migration:
1. Read the current schema. Don't reconstruct it from memory.
2. Every migration ships an up and a down — no one-way doors.
3. Run it against a copy of real data, not an empty test database.
4. One migration touches one concern. Never bundle two unrelated table changes.

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [
          { "type": "command", "command": "~/.claude/hooks/block-force-push.sh" }
        ]
      }
    ]
  }
}

#!/bin/bash
# Claude Code passes the pending call as JSON on stdin
cmd=$(cat | jq -r '.tool_input.command')

if echo "$cmd" | grep -qE '\-\-force.*(main|master)'; then
  echo "Force-push to main is blocked. Do it on purpose, not by muscle memory." 1>&2
  exit 2
fi
exit 0

#!/bin/bash
# Runs when Claude finishes a turn — checks evidence, not the summary
if ! grep -q "0 failed" test_output.txt 2>/dev/null; then
  echo "Gate failed: no test_output.txt with a real 0-failed line." 1>&2
  exit 2
fi
exit 0

---
name: reviewer
description: Reviews a diff. Never invoked to write one.
model: opus
---

You did not write this code. Assume it's wrong until you find evidence it isn't.
Check the diff against the stated requirement, not against what you'd guess the author meant.
Flag anything you can't verify — don't fill the gap with good faith.

Every one of those is a plain text file. None of it is exotic — the discipline is in wiring it up before you need it, not after a run has already gone sideways at 3am.

Anything that fails a gate or gets rejected on review doesn't get argued with. It goes back to be rebuilt. Nobody has to catch it by re-reading the whole transcript, because the transcript was never the thing doing the catching in the first place.

The honest way to describe where this leaves me: Claude Code has stopped being an assistant I supervise and started being an engineer operating inside a factory that has its own inspectors. I still write the spec. I still decide what gets built. I just stopped assuming that writing "and never do X" at the top of a file was the same thing as making it true.