Pruning the fleet

I sat down to cut a bloated fleet of specialist agents and found the opposite: it wasn't bloated. It was mildly broken in specific, fixable ways. Here's the framework for deciding when an agent earns its keep, and the honest caveats about what I couldn't measure.

I run a fleet of specialist subagents (a code reviewer, a few language-specific reviewers, a fixer, a planner, a quality gate that checks other agents' work, and so on). Sixteen of them. They'd accumulated the way these things do: one added for a real need, then another, until nobody could say with confidence which ones still pulled their weight. So I did a pruning pass. I expected to cut for bloat. I was wrong about the problem, and being wrong was the useful part.

The "earns its keep" test

I judged each agent on three signals, in order:

• Dispatch frequency: how often it actually got called, read from real session logs. Zero appearances is a strong hint, not a verdict (more on why below).
• Role redundancy: not "is this agent good?" but "is there a distinct decision-point where I'd ever choose it over what already exists on either side of it?" An agent with no slot of its own is overhead, however capable.
• Defect impact: is it corrupted, misconfigured, or quietly lying? Weighted by where it sits: a broken gate that runs on every job outranks a broken narrow reviewer.

A cut needed redundancy plus a dead usage signal. A repair needed only a confirmed defect, at any usage level. Those are different bars on purpose: you delete for irrelevance, but you fix for brokenness regardless of how often the broken thing runs.

What actually got cut: two, not a purge

Only two agents came out. A plan-only "orchestrator" that sat in the middle between two things that already plan and execute, pure connective overhead, and the logs showed it had never once been dispatched. And a documentation reviewer so narrow its single useful check folded cleanly into the general code reviewer; not worth its own routing decision. Both cuts rested on redundancy first; the zero-usage signal only corroborated.

What actually needed fixing: almost everything else

The real findings weren't bloat; they were quiet breakage that had been running for who knows how long:

• The quality gate that reviews other agents' work (the one with the most leverage, because it runs every time a subagent finishes) had a body that was truncated mid-sentence. It had been waving work through against incomplete criteria, and nothing flagged it because it still ran. That fix was priority one.
• A security-sensitive reviewer was quietly running on the cheapest model, while its peers doing equivalent-stakes work ran on a stronger one. Nobody updated the routing rule when the bar moved.
• A fixer agent had its verify commands hardcoded to one language's toolchain, so on a project in any other language it would report a clean pass it never actually observed. A fixer that can't verify the thing it's fixing is worse than no fixer; it lies green.
• An agent pointed at a helper that had since been archived, a dangling reference that's a silent no-op at best and an init error at worst. Another had a copy-paste artifact in a section heading. Small, but real.

The honest part: I couldn't measure what I most wanted to

The signal I'd have trusted most, real per-agent pass rates and latencies, wasn't available; the service that holds it wasn't reachable that session. So "earns its keep" leaned on dispatch frequency and role analysis, not measured scores. Worse, the dispatch data is itself coarse: the logs record activity at the session level, so "zero dispatches" really means "never surfaced in a session summary," not provably "never ran."

The caveat was on the table while I was deciding, and I decided anyway, because the two cuts stand on role redundancy, which holds independently of the usage numbers. The usage signal corroborates the argument; it doesn't carry it. And the cuts went to a staging branch, not a hard delete, so the call stayed reversible until the measured data backed it.

It did, later. A separate pass cross-checked the cuts against eighty-six real evaluation runs: the orchestrator had zero entries across all of them, confirming the dead-usage read; the doc reviewer had exactly one appearance, anecdotal at n=1, but consistent. The validation arrived after the decision and confirmed it rather than generating it, which is the right order: act on the defensible structural argument, then let the measurement catch up and check you.

The transferable lesson

Fleet audits usually find the wrong problem. The instinct is to count agents and cut for bloat, because that's the legible, satisfying move. But the real failure modes are quieter and don't show up in a headcount: a gate corrupted mid-file that still runs, a reviewer drifted onto the wrong model, a generalist with no routing boundary absorbing work meant for specialists, a reference pointing at something that no longer exists.

So the order matters. Audit for corruption and misconfiguration first; cut for redundancy second. A working fleet of fourteen beats a tidy fleet of twelve with a broken gate at the center of it. And when the measurement you trust most is missing, you don't get to skip the decision; you make it on the argument that doesn't need the number, stage it so it's reversible, and let the data confirm or overturn you on its own schedule. That last discipline is the one I keep relearning: the absence of a metric is not permission to guess, and it's not permission to stall either.