Your Retriever Returned 12 Chunks. The Agent Only Really Read 3 of Them.

A research agent pulls twelve passages, drops them into the prompt, and answers the question. The answer is wrong in a particular way: the correct fact was in the context, in passage seven, and the model wrote around it. Not a retrieval failure — the retriever did its job. A placement failure.

Position inside the context window is not neutral. Material at the very start and the very end of a long block gets used far more reliably than material buried in the middle. Once you accept that, "retrieve top-k and concatenate" stops being a reasonable default, because it treats slot 1 and slot 7 as interchangeable when they demonstrably are not.

🔍 The symptom, and how it lies to you

The failure looks like hallucination. The agent produces a fluent, plausible answer that contradicts a document sitting in its own prompt. Teams debugging this usually reach for the retriever first — tune the embeddings, raise k, add a reranker. Raising k often makes it worse: you have added more middle.

A cheap way to tell the two apart:

1. Take a failed query where you know the ground truth and know which chunk contains it.
2. Re-run with only that chunk in context.
3. If the answer is now correct, retrieval was fine. You have a placement or a dilution problem.

Step 3 is the whole diagnostic. Run it before touching the index.

Why "top-k concatenated" is the wrong shape

Most retrieval pipelines produce a ranked list and then lay it out in rank order, best first. That puts your second-best evidence at position 2 — a strong slot — and your third-best at position 3, and everything from rank 5 onward into the dead zone. But rank order is a claim about relevance, not about how much attention each slot will get. You are sorting one thing and being scored on another.

Worse, the low-ranked chunks are not free. Each one adds tokens, adds cost, adds latency, and adds distractor material the model may pattern-match against. A chunk that is topically similar but factually irrelevant is the most expensive kind of noise: it is close enough to look like an answer.

Three fixes, cheapest first

1. Retrieve fewer, harder. The strongest single intervention is usually to cut k and add a reranker so the survivors are genuinely load-bearing. Four dense, high-precision passages beat twelve mediocre ones — not because more context is impossible to use, but because you have removed the middle rather than tried to compensate for it.

2. Order for the edges, not for rank. If you must keep ten chunks, stop laying them out 1→10. Fold the ranking so the best material occupies both ends:

def edge_order(chunks_by_rank):
    """Best-ranked chunks land at the start and end; weakest sit in the middle."""
    head, tail = [], []
    for i, chunk in enumerate(chunks_by_rank):
        (head if i % 2 == 0 else tail).append(chunk)
    return head + list(reversed(tail))

# rank order:  [1, 2, 3, 4, 5, 6]
# edge order:  [1, 3, 5, 6, 4, 2]

Rank 1 opens, rank 2 closes, and the weakest material is where attention is thinnest anyway. This costs nothing and changes no other part of the pipeline.

3. Put the question after the context, not only before it. A long block of retrieved text between the instruction and the generation point means the instruction itself is now buried. Restating the actual question immediately after the documents — a single line, no ceremony — keeps the task in a strong slot.

[system instructions]
[12 retrieved passages]
Question, restated: which contract clause governs early termination?

Make the middle observable

The reason this bug survives so long is that nothing in the trace records it. The log shows twelve chunks retrieved and an answer produced. Nowhere does it show that chunks 5 through 9 contributed nothing.

Two instruments worth adding:

• Forced citation. Require the agent to name which passage each claim came from, and store that. Then compute a per-position usage histogram across a few hundred real queries. If positions 5–9 almost never get cited while the reranker insists they are relevant, you are looking directly at the dead zone.
• A planted-fact probe. Insert a synthetic, unmistakable fact at a known position — "the internal project codename for this migration is Ambergris" — and ask for it. Sweep the position across runs. This gives you a rough, honest curve for your model, your chunk size, your prompt shape. Do not import someone else's curve; it moves with model, context length and formatting.

⚠️ Measure with your own traffic. Position effects vary enough between models and prompt layouts that a published finding is a hypothesis about your system, not a description of it.

When more context genuinely is the answer

None of this argues for starving the agent. Some tasks — synthesizing across a whole document set, comparing many candidates — legitimately need everything in view, and the fix there is structural rather than positional: map over the chunks in separate calls, produce per-chunk extracts, then reason over the short list of extracts. That converts a wide-context reading problem into a narrow one, and the middle stops mattering because there is no longer a middle.

The rule of thumb: if the task is find and use a specific fact, retrieve less and place it well. If the task is aggregate over everything, don't place it at all — decompose it.

The takeaway

Stop treating the prompt as a bag. It is an ordered structure, and the order carries weight that your retriever's ranking never accounted for. Before you spend another sprint on embeddings, run the one-chunk test on a handful of known failures. If the answer comes back correct with a single passage in context, your index is fine — your layout is the bug.