Who Watches the Supervisor

In Exponential’s drive loop, a coder sub-agent writes code. This coder can’t grade its own work. That was an early design decision, borrowed from Boris Cherny’s loop engineering principle. Therefore the supervisor exists: a separate agent, with separate context, evaluating the output against the spec.

I spent most of my early effort optimizing the coder. Better context, more relevant files, clearer instructions. That helped, but it helped linearly. A ten percent better coder prompt produced roughly ten percent better first drafts.

The step change came when I redesigned the supervisor. A better reviewer didn’t just catch more bugs. It changed the trajectory of every subsequent attempt. The improvement compounded through the feedback loop. A mediocre coder paired with a sharp supervisor converges to the right answer, because the feedback corrects the drift round over round. A strong coder paired with a weak supervisor ships flawed code cleanly. Every time.

The quality ceiling of the entire system is set by the reviewer, not the builder: I was wasting my time optimizing the wrong agent.

What the reviewer sees matters more than how smart it is

The supervisor sees the spec and the diff. It does not see the coder’s reasoning, its chain of thought, or its plan. That’s deliberate.

A wrong answer with a plausible explanation gets more lenient treatment than a wrong answer standing alone. I noticed this in my own reviews first: if the walkthrough sounded thoughtful, I’d catch fewer issues in the diff. The reasoning was a distraction. It created anchoring bias. The supervisor has the same bias unless you remove the input that triggers it.

The review needs to be comparative. Does this diff satisfy this spec? That’s the only question. The spec is the contract. The diff is the deliverable. The review is the acceptance test. Everything else is noise that makes the noise harder to spot.

This design choice had a side effect I didn’t anticipate. It made the spec the most important artifact in the system. A vague spec produces vague reviews, which produce vague feedback, which sends the coder in circles. A precise spec produces precise reviews that converge in one or two rounds. The spec isn’t just input to the coder. It’s the reference standard the entire loop is calibrated against.

But who reviews the reviewer?

The discourse inside the AI community has caught up to this question in the past month. “Graph engineering” is the new term. The framing: single-agent loops drown in their own context. The agent that researched, drafted, and reviewed all in one loop ends up rubber-stamping its own output because it can’t be objective inside the same context that produced the work. The fix is to separate agents into specialized nodes with clean context, pass structured state between them, and route deterministically. You agentic system graduates from “while-loops” to “org charts”.

I read that and thought: we’ve been doing this all along. The coder and supervisor in xpo drive are separate agents with separate context by design. The spec flows between them as structured state. The Go orchestrator handles the routing. Although we never called it graph engineering, it’s the same architecture, arrived at by solving the same problems.

What graph engineering adds to the conversation is the question of scale. Carlos Perez frames it through Goodhart’s Law: a metric optimized hard enough stops measuring what it once did. A support team optimizing for ticket resolution speed starts deflecting customers instead of helping them. The metric improves. The thing the metric was supposed to measure gets worse. In agent systems, the same dynamic plays out. A coder optimizing for “tests pass” can learn to write trivial tests. A supervisor optimizing for “matches the spec” can learn to read specs loosely.

The graph engineering answer is: add more loops. A counter-metric loop watching what the primary loop is incentivized to ignore. A third loop making sure the first two aren’t just confirming each other. The graph emerges from the need for mutual oversight. That’s real, and it’s a useful way to think about system design.

But it overreaches when it suggests you can automate judgment all the way down. Perez himself supplies the caveat: a graph of loops will fail wherever it’s built without anchors. Agents watching agents can become circular, each confirming the others’ outputs with no contact with reality. His answer is “frozen nodes,” things the system reads but never edits. Ground-truth references that can’t be gamed by optimization pressure.

In xpo drive, the spec is the frozen node. The coder can’t change it. The supervisor evaluates against it but doesn’t rewrite it during the build cycle. The deterministic orchestrator, a Go function that owns lifecycle transitions, retry logic, and escalation, is another one. It doesn’t negotiate. Retries exhausted? BLOCKED. Timeout hit? Stop. These are edges no LLM is allowed to redraw.

But frozen nodes need someone to set them. Someone has to write and review the spec. Someone has to decide what “good” means, not “does it match the spec” good, but “is this the right thing to build” good. That’s why every issue in Exponential carries a spec and a walkthrough as first-order artifacts. The spec is the human’s input: what should be built, approved before any code runs. The walkthrough is the agent’s output: what it did and why, written for the human to read against the diff. Together they’re the review surface for the outermost loop, the one no agent runs. If the supervisor drifts, the walkthrough is where I catch it. I read the trail, adjust the prompts, and the next run is better. The AiSE process captures those learnings as reusable patterns, so the improvement compounds.

So in the end, no matter how many loops and frozen nodes you add, the thing that reflects what quality means for the product and that makes up the outermost loop: that’s a human. Not a bottleneck. The one that makes all the inner loops worth running.