Your Two Reviewer Agents Disagree. Who Breaks the Tie?

A contract-review pipeline sends each incoming vendor agreement to two agents. One checks it against the company's liability policy, the other against its data-processing policy. On a Tuesday afternoon, an agreement comes back with two verdicts: the liability reviewer says approve, the data reviewer says block — unlimited subprocessor rights.

The orchestrator has no rule for this. It takes the last message in the list and moves on.

That is the failure. Not the disagreement — the disagreement is the system working. Two reviewers looking at the same text and reaching different conclusions is exactly why you ran two. The failure is that nobody decided, in advance, what a split verdict means.

🔍 Most "consensus" designs never define consensus

The multi-agent literature is full of critic agents, verifier agents, and debate loops. What gets specified far less often is the resolution rule: given N outputs that don't agree, which one becomes the answer?

In practice, teams end up with one of these by accident rather than by choice:

• Last writer wins. Whichever agent finished last is the one whose output sits at the end of the message list, so it's the one the orchestrator reads. The tie-break is a race condition.
• First writer wins. The orchestrator short-circuits on the first non-empty result. Fast agents outvote careful ones.
• Concatenate and let the summarizer decide. Both verdicts go into a final "synthesize the findings" prompt. The model, having no instruction about precedence, usually splits the difference and produces something like "generally acceptable with some concerns" — a verdict neither reviewer gave.

That third one is the most dangerous, because it looks like judgment. A block and an approve went in; a maybe came out. Nothing in the trace flags that a disagreement occurred at all.

Four resolution rules that are actually decisions

1. Asymmetric precedence (any-blocks-wins)

The simplest rule, and the right default for anything with a safety or compliance dimension: the reviewers are not peers. A block from any reviewer is a block. An approve only holds when every reviewer approves.

verdicts = [r.verdict for r in reviews]
if "block" in verdicts:
    return Block(reasons=[r for r in reviews if r.verdict == "block"])
return Approve()

This is not consensus, and that's the point. When the cost of a false approve is much higher than the cost of a false block, you don't want an average — you want a veto. Use this whenever the reviewers have disjoint jurisdictions: the data reviewer is the sole authority on data terms, and the liability reviewer has no standing to overrule it.

⚠️ The cost is over-blocking. If one reviewer is noisy, its false positives now gate the whole pipeline. Track block rate per reviewer, not just overall — a veto rule makes each agent's precision a system-level property.

2. Adjudication by a judge agent

Give the disagreement to a third agent whose only job is to decide, with both verdicts and their cited evidence in front of it.

This works when the reviewers have overlapping jurisdiction — both are reading the same clause and reaching different conclusions about it. There's a real question to answer, and an escalation prompt with both arguments is a reasonable way to answer it.

It works badly in two situations. When the reviewers disagree because one of them didn't have the relevant text in context, the judge is adjudicating between an informed opinion and an uninformed one without knowing which is which. And when the judge shares a model and a system prompt with the reviewers, it tends to inherit the same blind spot that caused the split.

💡 Make the judge's output structured: winner, reason, and confidence. A judge that can only pick a side will always pick a side, including on splits that should have escalated.

3. Evidence-weighted, not vote-weighted

Don't count verdicts. Count citations.

Require every reviewer to return its verdict plus the specific span it relied on — clause number, quoted sentence, retrieved document id. Then resolve by checking the evidence rather than tallying opinions:

• Both cite the same clause, opposite readings → genuine interpretive dispute. Escalate (rule 4).
• Only one cites anything concrete → the uncited verdict loses. It's a vibe, not a finding.
• They cite different clauses → not a disagreement at all. Both findings are true and belong in the output together.

That last case is more common than it looks. Half of what registers as "the agents disagree" is really two agents answering two different questions and an orchestrator that flattened both into a single yes/no field. Widening the output schema resolves it without any tie-break logic at all.

4. Escalate the split itself

Route the disagreement to a human, and route it as a disagreement — both verdicts, both citations, the specific clause in contention. Not as a generic "agent needs review" ticket.

The economics usually work. Splits are a small share of volume, and they're pre-triaged: the reviewer that disagrees has already localized the argument to one clause. A human decides one narrow question in a minute instead of re-reading the contract.

The trap is escalating without the evidence attached. "The agents disagreed on contract #4471" is worse than useless — the reviewer now has to reconstruct the argument from scratch, and the escalation costs more than doing the review manually would have.

Choosing between them

These compose. A workable stack: evidence check first to strip out the fake disagreements, veto rule for jurisdictional blocks, judge for the interpretive remainder, escalate anything the judge returns low-confidence on.

The instrumentation that makes any of this real

Whatever rule you pick, log the split. One boolean field — agents_disagreed — plus which rule fired and which verdict won.

Without it, a system with a resolution rule is indistinguishable from a system with a race condition. Both emit a single confident verdict. The one that logs disagreements tells you, a month later, that two reviewers split on 8% of contracts and the judge sided with the liability reviewer nearly every time — which is a finding about your reviewers, not about your contracts.

✅ Decide the tie-break at design time, in code, where you can read it. ❌ Don't let it emerge from message ordering and a synthesis prompt that was never told disagreement was possible.

An agent that returns a confident answer to a question its collaborators answered differently isn't converging. It's just louder.