Dead Letters

Thirteen guards stand between my agents and the shell. Every command any agent runs on my machine passes through all of them: a destructive-command blocker, an egress guard, credential-read protection, a git-safety wall, and a supporting cast behind those. This June I did the thing I should have done the day I installed the first one. I replayed the traffic: 2,236 real logged commands from nine weeks of work, pushed back through every guard, every verdict recorded.

Twelve of the thirteen never fired. Not once, on anything real.

Dead letters, not dead code

The obvious reading is that the guards are broken, and it is wrong. Synthetic probes confirmed every one of them fires on a genuine trigger: the catastrophic delete is denied, the credential read is denied, the unapproved host is denied. They are dead letters for a different reason. The operator they guard never does the dangerous thing. Nine weeks of real traffic contained no credential reads, no secrets on command lines, no egress anywhere off the allowlist. A disciplined operator starves his own guards, and a guard layer is a threat model made executable, so what I had built, it turns out, was a museum of fears I had already trained myself out of.

Here is the trap in evaluating that museum: a guard that never fires has a near-perfect accuracy score. It also carries no information. Accuracy is the wrong instrument; what a guard owes you is measured on two axes, what it catches and what it costs, and I had only ever measured the first. The catch axis got two full adversarial tournaments, which I have written about: every bypass hunted down, thirteen shapes, three root causes. The cost axis, the false alarms and the latency and the friction, had never been measured at all. It could not have been. The layer keeps no record of denials; the log file that was supposed to hold them existed and had never once been written. Until the replay, the production behavior of my most-invoked safety mechanism was not just unknown. It was unknowable.

The one that fires

The census’s second finding is what the single live guard was doing with its monopoly.

The git-safety wall produced 19 denials across the corpus, 0.8 percent of traffic, the only guard with any real fire record. Fourteen of the nineteen were false. The rule blanket-blocks any commit or push while the working copy happens to be sitting on the main branch, which correctly stops a commit to main and also, incorrectly, stops the entirely legitimate push of a feature branch, a case a separate, better-targeted rule downstream already handles. Roughly three fires in four, wrong, and every false one lands on the disciplined path, the agent doing exactly what the branch policy asks.

So the whole layer’s production output for nine weeks totals five true denials, fourteen false alarms, and twelve guards’ worth of silence. I have argued that most of your findings are false and proposed scoring every instrument like an agent, catches minus false alarms. This was the first instrument I scored that way after writing it, and at its only live firing point, the false rate was 74 percent. The essay was not wrong by being too cynical.

The tax

The silent axis still bills. The guard chain costs about 460 milliseconds of compute per command as a floor, nearly twenty-nine thousand subprocess launches over the corpus, roughly 17 minutes of pure guard overhead in a heavy session. That is the premium, paid on every single command, for five true denials in nine weeks.

Insurance can be worth a premium. But two specimens from the study show the friction is not hypothetical. Mid-audit, the widest of the thirteen scanners blocked the audit itself: a pure read, a text search over the guards’ own directory, denied as a dangerous write, live, while we were measuring false positives. The layer false-alarmed on the instrument measuring its false alarms. And the egress guard, tested against a corpus of 68 known-good commands, fired on a grep whose search pattern merely contained the word curl, then, unable to find a hostname in a command that never had one, refused it on principle.

The bind

Every false positive above has an obvious fix: tighten the pattern. Exclude greps. Special-case the feature-branch push. Narrow, narrow, narrow, until the noise stops.

That is the one fix you must not take, and the reason is the census’s most useful lesson. The false alarms and the bypasses grow from the same root. These guards read the text of a command, while the shell executes its meaning, and everything lives in that gap. The guard that fires on curl inside a quoted search pattern is blind to position, and the same position-blindness is what an attacker walks through in the other direction, spelling a dangerous meaning in text the scanner does not recognize. One root, two harms. Narrow the pattern to kill a false positive and you have, with the same keystrokes, widened the bypass. The noise and the holes are not two problems. They are one property, observed from opposite sides.

The quieter version of the same lesson: four separate guards protect the credential surface, and all four share one list of protected paths. They diverge on a single question, whether to normalize what they read before matching it, and only one of them does. That one divergence class is what historically produced the three critical forgeries a fresh-eyed audit had to catch. Guards drift apart not at the policy, which everyone reviews, but at the preprocessing, which nobody does.

The other axis, instrumented

The census did not end in a teardown. The wall stays; the point was never that guards are worthless, it is that an unmeasured guard is unpriceable. What changed is that the cost axis is now instrumented, and the moves are transferable to any guard system, hooks or linters or review gates:

Give every rule both halves of a test. Each guard already had a must-block probe; each now needs a must-pass corpus, built from real passing traffic, replayed on every change. The benign corpus that caught the egress guard’s habit was 68 commands and an afternoon.

Write the deny log. A safety layer that records nothing when it fires cannot be scored, and a layer that cannot be scored converges on ritual.

Fix false positives behaviorally, never by narrowing. Parse position instead of scanning substrings; protect the resource instead of the spelling of the attack. The convergent posture on the catch axis turns out to be the convergent posture on the cost axis too, which is what you would expect of one root.

And retire the redundant. Three of my thirteen guards were strict subsets of other guards: each a paid subprocess launch on every command forever, and a seam where two copies of one policy can drift apart.

The limits

The zero fire rate is a fact about my habits as much as my guards; a sloppier operator, or a hostile one, would light this layer up, and twelve dead letters on my traffic says nothing about their value on someone else’s. The latency figure is a floor from a measurement that deliberately short-circuited one guard’s stateful bookkeeping. Four guards’ fire rates depend on runtime state a replay cannot reproduce, so their zeros are floors too. The benign corpus was 68 commands, representative rather than exhaustive. And a synthetic probe proves a guard can fire, not that it will fire on the variant that finally matters.

Two questions

The portable discipline fits in two questions, asked of every guard you run: when did this last fire, and when it fired, was it right? If nothing in your system can answer, that is the finding. My layer’s answer, once it existed, was that the only thing standing guard was mostly wrong, everything else was silence billed by the millisecond, and none of it had a ledger. A guard that never fires has perfect accuracy and tells you nothing. The false-alarm ledger is not overhead on the safety system. It is the safety system’s own health check, and mine had never been run.