Sermons vs. Instruments

There's a rule in my project configuration that says, in effect, never push to main. It's written in plain English, in a file the agent reads at the start of every session. It is a good rule, clearly stated, prominently placed, and the agent violates it periodically. Not from malice, not from carelessness: a written rule is a suggestion the model mostly follows, and "mostly" is the entire problem.

Run the arithmetic on "mostly." Suppose a control binds about 80% of the time, which is a generous estimate for prose guidance. Across a thousand agent actions in a month, that's two hundred lapses. Most of them cost nothing; the agent self-corrects, or the action was reversible, or you catch it in review. But the damage isn't distributed evenly, and the two hundredth lapse is the one that force-pushes over a colleague's work, or drops a production table, or commits a credential. You don't get to choose which 20% slips. You pay, every month, for the runs where the words didn't bind, and the bill is denominated in incidents, not averages.

One label before we go further, so these numbers don't wear more authority than they've earned: every percentage in this essay is a stipulated model, not a measurement. The hook logs and session transcripts to turn at least two rungs of the ladder into observed rates exist on my machine, and until I've done that exercise, treat the arithmetic as an argument about shape, not a result.

Now hold the same invariant a different way: a PreToolUse hook that inspects the command, sees git push origin main, and denies it. The agent cannot push to main. Not "usually doesn't." Cannot. The action is physically unavailable, and the 20% failure band collapses to zero, not because the agent got more obedient but because obedience stopped being load-bearing.

One of these controls is a sermon. The other is an instrument. The entire discipline of AI engineering, I'd argue, is the slow migration of behavior from the first kind to the second.

The two ways to make a system behave

In the agent era there are exactly two ways to make a system do what you want. You can tell it: advisory text, a prompt, a rule file, a CLAUDE.md, a system message. Or you can build a mechanism that makes the wrong behavior impossible: a hook, a type, a network topology, an evaluation gate. The first binds probabilistically. The second binds deterministically. My own notes put the gap baldly as a model of the failure mode, not a measured rate: "CLAUDE.md Is Advisory (~80%); Hooks Are Deterministic (100%)." Words land most of the time. Mechanisms land every time inside the contract they actually inspect, because they don't route through the model's judgment at all.

None of this is a knock on words. Words are how you communicate intent, taste, and the why behind a constraint, which no mechanism can carry. The mistake is using a sermon where an instrument is required. As the lesson file says: "Critical safety rules placed only in CLAUDE.md ... get violated periodically because Claude treats CLAUDE.md as advisory guidance, not a contract." The model isn't misbehaving. It's doing exactly what it was built to do: weigh context and respond plausibly. A rule in a prompt is one more piece of context to weigh, and context can lose.

The axis

Lay the common controls on a single line, purest sermon to purest instrument, and ask each one the only question that matters: where does it bind?

The percentages below are still the stipulated model, placed beside the claims they qualify. Replacing them with observed rates would require a separate log-reading lane.

Prompt instruction. "Please don't do X." Lives entirely in the model's attention, binds in the model at maybe 70–85%, and degrades as the context fills and the instruction recedes into the scroll. Cheapest to write, weakest to hold.

Rule file (a CLAUDE.md, a rules/*.md): a durable, re-injected prompt. Consistent, survives across sessions, and, in the stipulated model, binds in the same band, because it is still text the model chooses whether to honor. Same substance, longer half-life.

Exemplar. A worked example or golden transcript. Stronger than a rule, because it shapes the model's sense of what good looks like instead of asking it to obey an abstraction. Still probabilistic, but it moves the distribution: the model imitates more reliably than it complies.

Type contract. Here we cross into instrument territory. A type that makes the illegal state unrepresentable doesn't ask the model to remember a constraint; it removes that state from the space of expressible programs, and the compiler does the enforcing. Binding approaches 100% for the property the type captures, and only that property.

Hook. A PreToolUse deny, a pre-commit gate, a CI check. It intercepts the action at execution and refuses it. This is the canonical instrument: whatever the model intended, the mechanism sits between intent and effect. In the model, it binds at 100% on the action it inspects.

Topology. The strongest and most invisible instrument, because it makes the rule a property of the world rather than a check bolted onto it. Bind a service to a loopback-only interface and "only local callers may reach this" stops being a policy you enforce and becomes a fact of the topology. There's no rule to violate because there's no path to misuse. When you can arrange it, topology beats every check downstream.

Eval gate. A test suite, a scored benchmark, a regression harness that must pass before code ships. It binds differently than the others: not on a single action but on aggregate behavior over time. It's the instrument that catches the drift no hook anticipated, the slow erosion of quality that no single deny-rule sees. It's how you keep probabilistic components honest in the large.

Top to bottom, binding strength climbs, and cost-to-build climbs with it. A prompt line takes thirty seconds; a topology decision is an architecture. So the skill isn't "always pick the bottom of the list." It's knowing which band each control actually binds in, so you stop trusting an 80% control to hold a 100% invariant.

This is where the verification capital idea from the vault thesis comes in. Every instrument you build is stored, compounding trust: a hook written once keeps paying out on every future run, the way a test suite turns one act of verification into a permanent asset. Sermons don't accumulate; they're re-litigated every session, and sometimes they lose. Instruments are capital.

The honest counter: instruments have contracts too

Instruments aren't magic. They bind deterministically only within the contract they actually implement, and that contract is frequently narrower than the name on the tin.

The cleanest example I've hit is worktree isolation. You spawn parallel agents with isolation: "worktree", each in its own checkout, and you reason: good, they can't step on each other. For the property the mechanism implements, you're right. That property is narrower than you think. The lesson file is blunt: "Worktree isolation prevents merge conflicts, not logical overwrites." Two agents in two worktrees can both edit a shared config to mutually incompatible ends; git merges both cleanly, because the bytes don't collide, and you ship something neither agent intended. The instrument did its job. Exactly its job. Its job wasn't the thing you needed.

It gets better, by which I mean worse: instruments can have leaks in their contract. The same worktree primitive assumes agents write paths relative to their isolated cwd, and that guarantee leaks: "Parallel agents spawned with isolation: \"worktree\" are expected to make all their edits inside the isolated worktree. But when an agent writes using an absolute path (rather than a path relative to its worktree cwd), the write can resolve to the main repo directory instead. The change lands outside the worktree branch entirely." The isolation boundary is real, and an absolute path steps right around it. The wall you trusted turns out to have a door, and the agent walked through it without ever noticing there was a wall.

So the deterministic 100% in this essay's model is always conditional: 100% on the action the instrument inspects, the state the type captures, the path the hook sees. Miss the actual failure mode and you've built a beautifully reliable guard on the wrong door. That false confidence is worse than a sermon, because a sermon at least came labeled: you knew not to fully trust it.

And over-instrumenting is its own failure. Encode taste in a hook and you get a brittle gate that fights every legitimate exception and erodes trust until someone disables it, at which point you have zero controls and a story. Judgment calls, style preferences, the why behind a constraint: these belong in words, because their value lives in nuance a hard rule would flatten. My own notes land it: "test-impact hints beat 'do TDD' instructions; prefer building environment signals over writing behavioral rules." Even there the move isn't "write a TDD hook"; it's change the environment so the right behavior is the easy one. The art is matching the control to the band, not maxing out the instrument count.

A promotion heuristic

So when does a behavior get promoted from sermon to instrument? The rule I've settled on is one question about consequences:

Does this invariant have to hold every single time? If yes (one violation is an incident, a corrupted dataset, a leaked secret, an irreversible action), it must be an instrument. In the stipulated model above, words bind at 80%, and 80% is unacceptable for anything where the remaining failure band is catastrophic. Build the hook, encode the type, fix the topology. My lesson file says exactly this: "For invariants that must hold every time, write a PreToolUse hook that denies the disallowed action."

If no (the behavior is a preference, a default, a matter of taste or judgment where occasional deviation is fine and sometimes even correct), leave it a sermon, and enjoy the flexibility. The same lesson finishes the thought: "Reserve CLAUDE.md for preferences, judgment calls, style guidance, and explanations of why, not absolute prohibitions." A rule file is the right home for "prefer this pattern," "here's our house style," "this is why we do it this way." Force those into mechanisms and you get a rigid, resentful system that can't handle the exception the words would have allowed gracefully.

The framing that makes this whole field legible is the one in my agent-safety note: "Agent safety is the practice of giving AI agents useful autonomy inside explicit trust, permission, and evaluation boundaries." The boundaries are the instruments. The autonomy inside them is where the sermons, and the actual work, live. You're not choosing between trusting the agent and caging it. You're deciding, control by control, which boundaries must be walls and which can be words.

Measure your practice by the answer. A mature AI-engineering setup isn't the one with the most rules in its CLAUDE.md. It's the one where every invariant that must hold has quietly stopped being a rule at all: the prohibitions have become impossibilities, and the words are saved for the things words are actually good at. That migration, from sermon to instrument, one invariant at a time, is the work.

Goes deeper in the book → Chapter 3: The Guard That Enumerates Loses