Map-Reduce or Sequential Refine? How an Agent Should Read 400 Pages

A research agent is pointed at 60 vendor contracts — roughly 400 pages — and asked one question: which of these agreements let the counterparty sublicense customer data to a third party? The documents do not fit in one call, or they fit at a price nobody wants to pay per question. So the work gets chunked. And the moment you chunk, you have made an architectural decision, whether or not you noticed making it.

There are two common shapes. Fan the chunks out, process them independently, and merge the results. Or walk them in order, carrying a running answer that each chunk updates. Both are reasonable. They fail in completely different places, and the failure is usually invisible until someone checks the output against the source.

🧩 The two shapes, stripped down

# map-reduce: chunks never see each other
findings = await gather(*[extract(chunk) for chunk in chunks])
answer   = reduce(findings)

# sequential refine: state carries forward
state = EMPTY
for chunk in chunks:
    state = refine(state, chunk)
answer = state

Four lines each. The difference is whether information flows sideways between chunks or only forward through a bottleneck.

Why refine loses things

The running state in a refine loop is a lossy compression, applied once per chunk. By document 50, the answer is not a view of 50 documents. It is what survived 49 consecutive rewrites, each performed by a model that was told to keep the summary a manageable length.

Three specific consequences:

• Order dependence. Feed the same 60 contracts in a different order and you get a different answer. That alone should disqualify refine for any question where the corpus has no natural sequence.
• Early anchoring. Whatever pattern the first few chunks establish becomes the frame. If contracts 1–5 all have a clean sublicensing prohibition, the loop starts treating "prohibited" as the default and downgrades later exceptions to footnotes.
• Silent eviction. Nothing announces that contract 12's carve-out was dropped at step 31. The state just gets shorter and more confident.

⚠️ Refine also has a nasty operational property: a bad step poisons everything downstream. If the model hallucinates at chunk 37, chunks 38 through 60 refine on top of the hallucination. You cannot retry chunk 37 in isolation because chunk 38's input no longer exists.

Why map-reduce loses things

Map-reduce has the opposite blind spot: each chunk is processed by a model that has no idea the other 59 documents exist.

That is fine for "does this contract permit sublicensing" — the question is answerable inside one document. It falls apart the moment the real question spans documents: which of these agreements conflict with the master services agreement? Is the definition of "affiliate" consistent across the portfolio? No single map call can see a conflict, so no conflict is ever reported. The output looks complete. It is complete per-document and empty across documents.

The second failure is at the seam. A definition sits in Exhibit A, the clause that relies on it sits forty pages later, and your chunker split between them. The map call over the second chunk resolves the term by guessing.

🔍 The question that actually decides it

Do not ask which pattern is better. Ask one thing about the question being answered:

Can the answer be computed per unit and then combined, or does it require units to be compared against each other?

• Per unit, then combined → map-reduce. "Find every clause matching X." "Extract the termination date from each." "Flag documents that mention arbitration."
• Requires comparison or accumulation → the merge step has to do real reasoning, or refine.
• Genuinely narrative or chronological → refine earns its place. Building a timeline from a stack of dated correspondence is a job where order is the signal, and each new document legitimately updates a prior belief.

The version that usually wins: map to evidence, not to prose

Most teams reach for map-reduce, write a map step that returns a paragraph summary, and then wonder why the reduce step is vague. The problem is the intermediate format, not the pattern.

Have the map step emit structured evidence instead:

{
  "doc_id": "vendor-contracts/acme-msa.pdf",
  "clause_ref": "12.3",
  "verdict": "permits_with_notice",
  "quote": "Vendor may sublicense Customer Data to affiliates upon thirty (30) days written notice",
  "defined_terms_used": ["Customer Data", "affiliates"]
}

Now the reduce step is not summarizing summaries. It is reasoning over a table of citations, which fits in context even for 60 documents because each record is small. And critically, cross-document questions become answerable at the reduce step: the definitions of "affiliates" from all 60 documents are sitting side by side, so an inconsistency is visible for the first time.

This is the practical fix for map-reduce's blind spot. You do not need chunks to see each other. You need the merge step to see enough specifics to do the comparison itself.

Cost and latency point the same direction

Map calls are independent, so they run in parallel — wall-clock time is roughly one chunk plus the reduce, not sixty chunks. Refine is strictly serial by construction. For a 60-document corpus that is the difference between a query someone waits for and a query someone schedules.

Token cost is closer than it looks. Refine re-sends the accumulated state on every call, so its input grows as the loop runs; map-reduce sends each chunk once and pays a second time only in the reduce. Say each chunk costs roughly a cent to process — map-reduce lands near 60 cents plus one merge call, while refine pays 60 chunk costs plus 60 copies of a growing state. The parallel version is usually cheaper and always faster.

Takeaway

Pick refine when order carries meaning and the answer legitimately evolves — timelines, narratives, running ledgers. Pick map-reduce for everything else, but do not let the map step return prose. Structured evidence with citations is what makes the reduce step capable of the cross-document reasoning people assume they are getting. The most common architecture failure in document agents is not picking the wrong shape. It is picking map-reduce and then throwing away, in the map step, the exact detail the merge step needed.