A Second Pass That Asks the Same Question Finds the Same Bugs. Change the Unit, Not the Temperature.

You added a second review agent to catch what the first one missed. It runs on the same artifact, with the same instructions, at a higher temperature. It reports four findings. Three are the same three the first pass reported, and the fourth is a reword of one of them. Review now costs twice as much and covers almost the same ground.

This is the normal outcome, not bad luck. Understanding why tells you exactly how to build a second pass that is worth its cost.

Why the second pass agrees with the first

A review pass is not general intelligence pointed at an artifact. It is a filter, and three properties define the filter's shape:

• The question — the defect vocabulary it hunts for. "Flag risky clauses" becomes a search for clauses that look risky.
• The unit of analysis — what counts as one item to judge. A clause? A sentence? A pair of clauses? The whole document?
• The view — which bytes it actually reads, and in what order.

Resampling changes none of those. Raising the temperature re-rolls the dice inside a frame that has already decided what is findable. The things pass 1 missed are usually not the items it was too sloppy to catch. They are the items its frame cannot represent.

That distinction is the entire design problem. A second pass earns its cost only if it is wrong in a different direction than the first.

Worked example: a contract review agent

Pass 1 is prompted: "Read this vendor agreement and flag clauses that create risk for us." It returns uncapped indemnity, an auto-renew with a short notice window, and a broad IP assignment. Genuine findings, and the kind any clause-by-clause reader produces.

Now the misses, sorted by why they were missed:

• "Confidential Information" is defined in §1 and used with a narrower meaning in §12.
• §7 obligates us to "provide reasonable access" with no deadline and no named owner.
• §4 grants termination for convenience; §19's minimum-commitment schedule makes exercising it expensive. Each clause is unobjectionable alone.

None of these is a risky-looking clause. The first needs the defined term as the unit. The second needs an obligation as the unit, with slots for owner and deadline so an empty slot becomes visible. The third needs a pair of clauses as the unit. Re-reading clause by clause a thousand more times will not surface any of them.

A second pass built as a genuinely different filter:

pass_1:
  question: "which clauses create risk for us?"
  unit:     clause
  view:     full document, in order

pass_2a:
  question: "list every defined term, then every use of it.
             Where does a use fall outside its definition?"
  unit:     defined term
  view:     definitions section + every occurrence, out of document order

pass_2b:
  question: "extract every obligation as (who, what, by when, measured how).
             Which rows have an empty slot?"
  unit:     obligation
  view:     full document

These are not a second opinion. They are a second instrument. Their findings barely overlap pass 1's because they cannot express most of pass 1's vocabulary, and that is the point.

⚠️ The anchoring leak: hide or show?

There are two jobs here, and conflating them wastes a pass.

• A discovery pass must not see the first pass's findings. Show them and it drifts into confirming them, spending its budget re-arguing the indemnity clause and closing with "I concur, plus one minor note."
• A verification pass must see them, and ideally only them, with the instruction to attack: "give the strongest reason this is not a real problem." That frame is cheap and catches false positives the discovery frame has no incentive to catch.

The common mistake is one pass asked to do both. It will do verification, because confirming a list in front of it is far easier than searching an empty space.

🔍 Measuring whether pass 2 is actually new

Track a unique-find rate per pass: findings no other pass reported, divided by that pass's total findings.

Suppose pass 1 reports six items. Pass 2a reports four, all four unique. Pass 2b reports five, three unique. Those are working instruments. A pass that reports five items with zero unique is a duplicate — retire it or re-frame it, but stop paying for it.

Plotted over time this becomes a design tool rather than a metric. A pass whose unique yield decays toward zero means either the artifacts changed shape or whatever generates them learned to avoid that defect class. Either way, the question needs to move.

Pair it with a second number: how many of the unique finds survive verification. High unique yield with poor precision is a noise generator wearing a reviewer's badge.

Designing the second pass

1. Write down pass 1's question, unit, and view explicitly. If you cannot state them, you cannot design a complement to them.
2. Change at least two of the three. Changing only the question usually leaves the unit intact, and the unit is what most often does the blinding.
3. Name a defect class pass 1 cannot see even in principle, then build the unit so that class becomes a visible property — an empty slot, a mismatch, a contradictory pair.
4. Decide discovery or verification, and wire the input accordingly. Never both in one pass.
5. Keep each question narrow and single-class. Broad questions collapse back into "look for obvious problems," which is pass 1 again.
6. Require evidence in a fixed shape — the quote, the section reference, the missing field — so overlap between passes can be computed mechanically instead of judged.
7. Measure unique-find rate and act on it.

The takeaway

"Review it again" is a hope, not a strategy. Coverage comes from running several filters whose blind spots do not line up, and blind spots are set by question, unit, and view. Before you add a second pass, write down what the first one cannot see even in principle. If you cannot name that, the second pass will find what the first found, and you will pay twice to learn nothing.