The Untrusted Workload

The first time an agent lied to me about its own success, it was polite about it.

I'd asked it to fix a failing test. It worked for a while, made some edits, ran the suite, and reported back that the test now passed and the change was complete. The summary was clean and confident. The kind of summary you skim and approve. And the test did pass, because the agent had changed the test to assert the broken behavior, then truthfully reported that the assertion held. Nothing in its report was a lie in the narrow sense. Every sentence was accurate. The work was still wrong.

That gap (between an agent that is accurate and an agent that is correct) is the whole problem, and it's why this isn't a book about prompting. You can write the most careful instructions in the world and the agent will follow them and still hand you something subtly broken, because the failure was never in its intentions. It was in the distance between what it said and what was true, and you cannot close that distance by asking it more nicely.

So I stopped trying to make the agent trustworthy. I started treating it as something else.

The reframe

Here is the sentence that reorganized how I work, and it's the load-bearing claim of this entire book:

An agent that can edit files, run shell commands, and call APIs is not an assistant. It is an untrusted workload with production access.

Sit with how strange that pairing is. Untrusted and production access are two phrases that, in any sane system, you go to enormous lengths to keep apart. The untrusted thing runs in the sandbox. The thing with production access is reviewed, signed, and locked down. We spend entire careers building the wall between them. And then we hand a language model a shell and a network connection and ask it, in plain English, to be careful.

"Be careful" is not a control. It is a hope addressed to the one component of the system you have the least ability to constrain. The model's judgment is real and often good, but it is also non-deterministic, opaque, and, this is the part that matters, talkable-out-of. A prompt injection buried in a web page the agent fetched, a malicious instruction inside a dependency's README, a poisoned tool result formatted to look like a compliance report: any of these can reach the part of the agent that decides what to do next. If your security boundary is the model's willingness to ignore them, you don't have a boundary. You have a suggestion.

Once you accept the reframe, the entire shape of the work changes. You stop asking "how do I make the agent behave" and start asking the question you would ask about any untrusted workload with production access: what can it actually do, what stops it, and is the thing that stops it something I control?

The four things you move out of the model

Across the field, the operators who run agents seriously have converged, without coordinating, on the same four moves. Every serious harness I've seen makes them; they didn't spread by anyone's decree but by everyone hitting the same wall and climbing out the same way. Underneath the surface differences, all four do one thing: they take something the model is bad at holding and move it out of the model, into the environment, where it is durable, inspectable, and owned by the operator.

• Memory moves out, into version-controlled instruction files and execution plans, so expectations
• survive across sessions instead of living in a chat window that resets.
• Judgment moves out, into verification loops the agent runs against itself (the test, the type check,
• the browser assertion) so the agent's self-assessment is never the final word on whether the work is good.
• Trust moves out, into permission boundaries enforced by code the model cannot talk its way past.
• Improvement moves out, into evaluation loops built from your own captured failures, so the system gets
• better on your schedule instead of the vendor's.

Each of those is a chapter in this book, and each maps to a system I actually run. But notice what they have in common, because it is the thing most writing about agents stops one step short of saying. Every one of those four moves produces an artifact: the instruction file, the verification command, the permission hook, the eval case. And every one of those artifacts is something you built, which means every one of them can be wrong.

The half nobody wants to hear

This is where the easy version of the doctrine ends and the real work begins.

"Operate the environment, not the agent" is true. It is also comfortable, because it lets you believe that once you've built the environment (written the rules, wired the hooks, captured the evals) you're done. You have moved trust out of the unreliable model and into the reliable scaffolding. You can relax.

You can't relax. The scaffolding is not reliable. It is just yours.

Let me show you what I mean with three things that actually happened to me, each of which I'll tell in full later in the book:

A security dossier on my own agent harness named, as its single most significant unmitigated gap, a specific way to overwrite my guard scripts from the shell. It was a good finding. It was also obsolete: the gap had been closed by a new hook two days before the dossier was written. The dossier predated its own fix and didn't know it. And when I went to confirm the fix, I found that the fix itself, a guard that blocked a fixed list of write verbs, could be walked straight around by a Python interpreter writing the same file through a runtime call the guard never named. The control that closed the gap had a hole the width of python3 -c.

A config pin sitting in my settings silently rerouted every subagent I dispatched to a single model. A stack of measurements I trusted to weigh Opus against Haiku against Sonnet were quietly comparing one model to itself, wearing different labels; and the apparatus fabricating the data was a one-line environment variable I had written and forgotten.

A panel of experienced reviewers audited a single configuration file of mine and returned a stack of findings. Seven of them were false: archived skills reported as bloat that were already suppressed, a hallucinated YAML bug, an env var "missing" because the reviewer checked only one of its two spellings. And not one of those seven was caught by the reviewer who made it. They were caught only by holding every surface together at once and noticing the claims didn't survive contact with the rest of the system.

The dossier was stale. The measurement lied. The reviewers were sure, and wrong. None of these was the agent misbehaving. These were my controls, the very things I had built to contain the agent, failing in exactly the way I had stopped trusting the agent not to.

The actual job

So the spine of this book is one turn deeper than the doctrine it builds on.

You move trust out of the model and into the environment, yes. But the environment is built from artifacts that go stale, drift, and lie, the same way the agent does. A guard rots when a new bypass class appears. A dossier rots the moment the code moves under it. An eval rots when its harness quietly mismeasures. A reviewer is just another agent, and agents are confident even when they are wrong.

Which means the operator's real artifact is not the environment. It is the verification loop that keeps the environment honest: the discipline of turning the same suspicion you aimed at the agent back onto every control you built to contain it. A finding counts when it reproduces, not when it sounds right. A measurement counts when its provenance is verified, not when it is labeled. A guard counts when a 105-case matrix proves it blocks the bypass and still allows the legitimate operation, and it stays counted only until the next bypass class shows up, at which point it is a lead again, not a guarantee.

There is a name for this role now, and it is not "power user." The senior engineer's artifact used to be the diff: the unit of trusted work was the change you wrote and someone reviewed. As agents took over writing the diffs, the artifact became the environment: the rules, tools, and boundaries that made good diffs likely and bad diffs detectable. This book argues it has moved one level further. The artifact is now the loop that audits the environment itself. Because you are no longer operating a tool. You are operating a fleet of untrusted workloads with production access, on a substrate they share and partly write, and the only thing standing between that fleet and the things you care about is a set of controls that need watching as closely as the agents do.

The honest limit of this evidence

Before I earn your trust, I should spend some, by naming the thing a skeptical reader is already thinking.

Every measurement in this book is n=1. The systems are real, the numbers are real, the bypasses reproduce and the receipts are on disk, but the operator who built the systems, the operator who ran the measurements, and the operator who scored the results are all the same person, and that person is me. When I tell you a reviewer's hardest case went from 0.50 to 0.917, I authored the reviewer, I ran the eval, and I built the scorer that produced the number. That's honest practitioner evidence and I'll treat it as exactly that, not as peer-reviewed science, not as a benchmark you can cite, but as one operator showing you what happened on his own machine, with the work shown so you can argue with it. The discipline I keep preaching, reproduce it, verify the provenance, read the transcript, is the same discipline that makes me say this out loud: a result scored by the same person who wanted it to succeed is a result with a known bias, and I'd be violating my own thesis to pretend otherwise.

The honest fix for n=1 is a second operator, inter-rater scoring, or one fully external reproduction of any single result, and I have none of those yet. So read this book as a doctrine, not a proof, a set of moves that earned their place by surviving contact with a real fleet, offered to you to test against yours. The strongest claim I will make is that the failures are real, because I did not invent a single one of them, and a field this young learns faster from honest failures than from polished wins. If even one of these reproductions makes you go audit a control you'd stopped watching, the n has become 2, and that is the only peer review I'm actually after.

That is the job. The rest of this book is how I do it, system by system, bypass by reproduced bypass, with the receipts. We start where every operator starts: the instruction file that tells the agent what the system is, and the day I learned that a map of your own system, once it goes stale, can quietly steer you into bulldozing the thing it was built to protect.