No Sermons in the Seal
The oldest rule in my notes says words bind at about eighty percent and mechanisms at a hundred. This is the methodology note for applying that rule to trust claims themselves: what it took to measure the gap instead of stipulating it, and why the seal that carries the result trusts a digest equality, never a sentence.
The oldest load-bearing rule in my notes is a stipulation, not a measurement: words bind at about eighty percent, mechanisms at a hundred. Sermons vs. Instruments runs the full argument and is careful about its own epistemics. The eighty is a model of the failure mode, a generous estimate for how often prose guidance holds, not a number anyone measured. The hundred is the honest half: a mechanism that inspects the action at the moment it is attempted binds every time inside the contract it actually inspects, because it never routes through anyone’s judgment.
This note is the methodology behind the verification spine, the page on this site where one harness config walks from a benchmark measurement to a ledger record to a portable seal. It answers two questions the page itself only demonstrates. What would it take to measure the gap instead of stipulating it? And once you have the measurement, what carries the claim to a stranger without turning back into a sermon, this time about instruments?
The measured half
You cannot measure “words bind at eighty percent” in general; it depends on the model, the words, and the day. What you can measure is the harness-level version of the question: given a config that claims to block destructive commands, what actually fires when the command arrives?
That is what HarnessBench does. It fires a corpus of destructive-command probes at a harness config: probes the mechanism must block, and ordinary honest work it must let through. The score is the Enforcement Effectiveness Score, true-block rate minus false-block rate. The subtraction is the point. Block nothing and you score zero. Block everything and the second term cancels the first, and you score zero again. There is no credit for paranoia and none for narration, because the model’s account of what it would have done never enters the arithmetic. Only the mechanism’s behavior does.
Three configs sit on the public ladder, and they are the argument in miniature:
- advisory, the rules written as prose in the config: 0.000. Not approximately zero. Zero. Every attack probe walks straight past the promise.
- naive-regex, a pattern match on the command string: +0.091. It catches half the attacks and punishes honest work almost as often, so the subtraction eats nearly everything it earns.
- semantic-clean-room, a mechanism that parses what the command actually does: +1.000. Every attack blocked, every safe command through. And the guard is independent of the corpus that scores it, so the ceiling is earned, not circular.
One honesty caveat, because the symmetry is tempting. The advisory rung’s zero is not the eighty measured down to nothing. The eighty models a language model reading your rule and usually honoring it. The zero is what a config’s prose does when nothing reads it at the moment of action. Different failure, same lesson: a sentence is not a mechanism, and the only question that matters is what fires when the command arrives. After the benchmark, the eighty in my rule is still a stipulation. The hundred is a measurement with a confusion matrix attached.
The claim wants to travel
Once you have the number, you want to cite it. A leaderboard on the benchmark’s own repo convinces nobody who does not already trust me, and “trust me” is the exact sentence this whole program exists to retire. So the result gets packaged as a seal: a signed attestation, in standard formats (in-toto, DSSE), that a stranger can verify with tools I do not control.
Here is the threat model that shaped it. The interesting adversary is not someone forging my signature; signatures are solved. The interesting adversary is the producer of the seal, which is to say me, signing an honest signature over a forgeable claim. Suppose the seal said, in words, “this config passed HarnessBench, enforced.” A verifier that accepts that sentence is trusting a name, and names are free. Rename any check to match the expected one, point it at any corpus, and the claim “covers” whatever I need it to cover. The signature would still verify perfectly, because a signature proves who said something. It never proves the thing said. A verifier that trusts the predicate’s prose has reintroduced words at exactly the point where words were supposed to have been retired.
So the seal’s enforced_proof refuses to read names. It resolves only when three digests, computed by three different tools over three different artifacts, turn out to be the same number: the digest of the sealed subject, the config_ref inside the signed check, and the config_sha256 the benchmark report computed over the config it actually measured. That equality cannot be asserted, only exhibited. If I had measured one config and sealed a claim about another, no wording could paper over the mismatch: the digests would simply differ. The spine page keeps a deliberately forged seal next to the real one, identical except for one flipped byte of config_ref, so you can watch the refusal happen in your own browser.
Honest by construction
Two more mechanisms keep the seal from claiming more than its evidence. Each check renders at a trust floor: the minimum of how strongly the check binds and what grade of evidence backs it. An enforced check on Grade-B evidence displays as “surfaced,” never as a bare “enforced,” because the weaker of the two is what a stranger should believe. And the verifier reports two verdicts, not one: authentic (the signature and freshness hold, so I said this, recently) and checks passed (the proof actually resolved). Collapsing them would let a valid signature launder an unproven claim.
One boundary stays open, stated rather than papered over: a seal asserts presence, never absence. It can prove a named check ran and what it found. It cannot prove no check was missing, and a receipt that claimed otherwise would be unfalsifiable. That claim is deferred until it can be made mechanically or not at all.
The recursion
The rule applies to its own paperwork. A README that says “enforced” is a sermon. A seal whose verifier trusts the predicate’s description is a sermon with a signature on it. And this essay is a sermon too: everything I have written here binds you at whatever rate my words bind, which is precisely why the page it describes does not ask you to take any of it on my word. Of all the sentences on that page, exactly one is written in the hundred-percent channel, and it is not a sentence at all. It is three digests agreeing.
Read the note, then go flip the byte.