Your Critic Agent Approves Everything Because It Read the Worker's Reasoning. Starve It of the Argument, Give It Its Own Evidence.
Forward the worker's transcript to your reviewer agent and it will approve nearly everything — anchored by a conclusion it treats as evidence. Here is the information diet, the mutation test, and the disagreement policy that turn a rubber stamp back into a check.
A reviewer that sees the defence before the evidence is not a reviewer. That is the whole bug, and it is wired into most critic-agent setups by accident: the worker finishes, its full transcript gets passed to a second agent labelled "reviewer", and from then on the review step returns approved on almost everything. The pipeline looks like it has two-stage verification. It has one stage and an echo.
Here is the shape it takes in an internal HR assistant. A writer agent is asked how much parental leave a UK employee accrues while on leave. It retrieves a handbook page, misreads a clause, and drafts an answer that is confidently wrong. Its trace ends with a line like "confirmed in the handbook, Leave & Absence section." That trace is handed to the checker agent along with the draft. The checker reads a claim, then reads a sentence asserting the claim was confirmed, and outputs: "Verified against the handbook. Approved." It never fetched the handbook. Nothing in its context told it to doubt a colleague who had already done the work.
Why the critic folds
Three mechanisms stack, and each one alone is usually survivable.
- Shared context means shared error. If the worker retrieved the wrong page, and the critic's only view of the source is the worker's quotation of it, then the critic is checking an answer against the same mistake that produced it. Consistency is guaranteed; correctness is untouched.
- A stated conclusion is an anchor. Models are strongly steered by assertions already present in the context. "I confirmed X" reads as evidence, not as a claim under review. The critic's job silently changes from is X true to is this reasoning plausible — and bad reasoning that reached a fluent answer is usually plausible.
- Agreement is the cheap action.
approvedcosts one short sentence. Disagreement requires naming the fault, locating it, and justifying it. Absent an explicit cost on false approval, the low-effort output wins, and it looks identical to a genuine pass in your logs.
⚠️ The failure is quiet by construction. Approval rates near 100% get read as "the worker is good," not "the critic is asleep."
Rebuild the critic as an independent verifier
The fix is not a sterner prompt. It is a different information diet. Build the critic's context from scratch rather than forwarding the worker's:
worker context: critic context:
task spec task spec (same)
retrieval results — (omitted)
scratchpad / reasoning — (omitted)
tool call history — (omitted)
draft answer draft answer (claims only)
its own retrieval (fetched fresh)
Concretely, four changes:
- Pass the artifact, not the argument. The critic receives the task and the output. Not the scratchpad, not the tool log, not "I verified this." If a claim's support is not in the artifact itself, that absence is a finding.
- Give it its own tools. The critic must retrieve, query, or run the check itself. Where possible, route it down a different path than the worker used — a different index, a direct database read instead of a search API, the actual file instead of a summary. Independent evidence is what makes the second look worth paying for.
- Force a per-claim verdict with a citation. Not a holistic score. Decompose the output into checkable claims and require, for each,
supported/contradicted/not-foundplus the retrieved span that justifies it. Asupportedwith no span is an invalid response and should be rejected by your schema, not read as a pass. - Make the null result speakable.
not-foundmust be a legal, blame-free verdict. If the only options are approve and reject, a critic that cannot locate the source will guess — and guessing resolves toward agreement.
🔍 Test that your critic can say no
Do not assume independence; measure it. Build a small mutation set and run it before you trust the stage:
- Take outputs you know are correct and corrupt them — flip a number, swap an entity, change a date, delete the qualifying clause from a policy sentence.
- Feed each corrupted artifact through the critic exactly as production would.
- Count how many it catches.
A critic that catches obvious numeric flips but waves through deleted qualifiers is telling you something precise about where its attention is, which is more useful than any aggregate pass rate. And run the same set with the worker's reasoning trace re-attached: if catch rate drops sharply, you have measured your own anchoring bug, in your own pipeline, with a number you can put in a regression test.
Watch the disagreement rate in production, too. It should be low but never zero. A critic stage that has not rejected anything in a week is not a quality signal — it is an untested code path.
When they disagree
An independent critic creates a problem the echoing one never did: genuine conflict. Decide the policy up front, because "have them discuss it" tends to end with whichever agent speaks last.
- One rebuttal round, bounded. Worker sees the specific
contradictedclaims and its spans, and may revise or defend once with new evidence. Not a debate loop. - Evidence beats fluency. If the critic cites a retrieved span and the worker cites only its own reasoning, the critic wins by default.
- Second disagreement escalates. Unresolved after one round → the artifact goes to a human with both verdicts and both spans attached, or the task fails closed. Do not spend tokens converging two agents on a question neither can settle.
The underlying principle generalises past text: a checker is only worth its cost if it can reach a different conclusion than the thing it is checking. The moment you hand it the worker's reasoning to "give it context," you have removed the only property that made it a second opinion — and bought yourself a review stage that ratifies mistakes at twice the price.