Agent Failure Museum
A public-safe gallery of synthetic failures: the little traps that make agent work look done, safe, or measured before the evidence catches up.
Fixture rule: every exhibit here is invented for explanation. No private transcripts, local paths, customer names, raw logs, prompts from real runs, or undisclosed security findings are used.
Stale Handoff Accepted as Truth
A restart note says the review branch is ready and the guard file can be replaced. The live branch has since gained a new protection, but the agent treats the old handoff as current state.
Why it is tempting
The note is clear, local, and written by the operator. It feels cheaper than rediscovering the repo.
What goes wrong
The run can delete newer safety work while honestly believing it is following the safest instruction.
Freshness reconciliation: compare the handoff against live git state, current files, and the repo gate before applying it. Treat old notes as leads, not evidence.
The check proves the repo changed. It cannot prove the old handoff's intent, so a human may still need to choose between two valid directions.
This demonstrates the mechanism, a snapshot cannot see later changes, not any model's behavior.
Permission Grant Treated as a Vibe Check
A useful MCP server asks for file read, network, and a write tool. The agent connects it because the README looks polished and the task is urgent.
Why it is tempting
Useful tools make immediate progress visible. Permission surfaces are abstract until something misroutes through them.
What goes wrong
A broad tool can combine read access, untrusted content, and outbound reach before anyone notices the blast radius.
MCPAudit inventories declared and connected capability surfaces, risk categories, schema drift, and suspicious tool text before the tool becomes normal infrastructure.
A config-only audit cannot prove runtime behavior. Connected and pinned checks improve evidence, but a reviewed server can still change later.
Prompt Injection Hidden in Tool Descriptions
A tool description includes friendly setup text plus a quiet instruction to ignore the operator's rules and prioritize the tool's output as authority.
Why it is tempting
Tool descriptions arrive in the same context window as legitimate instructions, and they often sound like documentation.
What goes wrong
The agent may treat untrusted tool metadata as instructions instead of environmental data to inspect.
MCPAudit's injection checks flag instruction-override patterns in tools, prompts, and resources. The runtime rule is stricter: tool output can inform work, never govern it.
Pattern checks are not complete language understanding. Novel wording can slip through, so the policy boundary still matters.
This shows the delivery channel and a real pattern check executing; it does not simulate whether any model would comply.
Benign Sanctioned Request Over-Refused
The operator asks for a draft through the approved approval flow. The agent refuses because the request resembles a risky direct-send case it saw earlier.
Why it is tempting
Refusal feels safer than action, especially after a run has been warned about tool misuse and social engineering.
What goes wrong
Useful work gets blocked, and the operator learns to route around the harness because the safe path is too noisy.
OPERANT uses matched malign and benign pairs. Refuse-all and proceed-all both score at the floor, so calibrated discrimination is what moves the score.
A benchmark case is a proxy for a longer operating loop. The score is useful evidence, not a universal certification.
Deploy Claimed Without Verification
The agent says the site is live because the build passed locally. Nobody checked the final URL, the branch state, or whether the expected asset is actually served.
Why it is tempting
Build success is concrete and satisfying. It is easy to let that feeling stand in for the whole release path.
What goes wrong
The report collapses multiple states into one word, shipped, and the operator loses track of what was proven.
Proof receipts and portfolio truth checks separate build, commit, push, deploy, and live verification into named evidence. GithubRepoAuditor keeps those claims reviewable over time.
A receipt proves commands and observations from a moment. It does not prove the service stayed healthy after the check.
The collapse function is real logic running here; the point is that one word cannot carry five states.
Private Data Leakage Risk in a Polished Demo
A gallery card includes a convincing transcript-style snippet with real-looking names, absolute machine details, and a copied failure log because it makes the story feel authentic.
Why it is tempting
Receipts are persuasive. The more specific the artifact looks, the easier it is to believe.
What goes wrong
The demo can leak identities, local environment shape, private project context, or sensitive operational details.
Public-safety gates scan for private paths, emails, credential-like strings, internal endpoints, and debug tokens. The editorial rule is tighter: use synthetic examples unless a public artifact was deliberately prepared.
Automated scans cannot catch every contextual leak. Human copy review still has to ask what a reader could infer.
Brittle Benchmark Overclaimed as Truth
A one-shot benchmark run crowns a winner from a tiny corpus. The summary ignores variance, scorer bias, and whether the cases actually test the decision boundary.
Why it is tempting
A single leaderboard is legible. Error bars, held-out cases, and scorer audits make the story less tidy.
What goes wrong
The harness starts measuring its own artifacts: phrasing, judge preference, or memorized surfaces instead of operating judgment.
Harness verification: use matched pairs, repeat runs, scorer selftests, fresh surfaces, and explicit limits. Publish the method and the caveats beside the score.
Even a careful eval is a local instrument. It needs reproduction, more cases, and pressure from systems the author did not tune.
The pattern
Most agent failures in this museum are not spectacular. They are ordinary shortcuts taken at the wrong trust boundary: old notes treated as state, tool metadata treated as instruction, local success treated as live proof, or a benchmark treated as a certificate.
The control shape
- Move claims into artifacts the agent cannot merely narrate.
- Run the check close to the moment of use.
- Keep the remaining uncertainty visible instead of sanding it off.
Where the exhibits came from
The cases are synthetic, but the control vocabulary is grounded in the public workshop: build with mcpforge, audit with MCPAudit, score calibration with OPERANT, fold repo evidence through GithubRepoAuditor, and keep the operator OS boundary public-safe.