Budgeting the Context Window: Dividing a Fixed Space

A research agent with a large context window kept losing its constraints around turn twenty. The window wasn't full. The problem was that nobody had decided how the space was divided, so it was allocated by accident — in arrival order, first come first served, with the accumulated history quietly consuming whatever the earlier claimants hadn't taken.

Context is a budget with several competing claimants. Left implicit, the allocation is decided by whichever tool returned something large.

The six claimants

Every turn, these compete for the same space:

1. System prompt. Fixed, small, stable. Should be a page.

2. Tool definitions. Fixed per phase. Grows with the tool count, which is one more argument for a curated list.

3. Structured run state. The work ledger, constraints, decisions. Small, regenerated each turn by the loop.

4. Retrieved material. Variable, and the biggest discretionary lever.

5. Accumulated history. Prior turns and tool results. → Grows without bound by default, which is why it ends up eating everyone else's allocation.

6. The current turn. The immediate task and any constraint restated before acting. Small, and the most attended position.

Set the budget explicitly

Pick a working budget well below the window limit — leaving headroom is what prevents the emergency compaction that drops something important — then allocate:

working budget: 32,000 tokens

system prompt        1,000   fixed
tool definitions     2,000   fixed per phase
run state            1,500   capped, regenerated
retrieved material  10,000   hard cap
history             16,500   residual — compacts to fit
current turn         1,000   always reserved

⚠️ Note that history is the residual, not a claimant with its own allocation. That single choice is what makes the scheme work: everyone else takes their fixed share, and history compacts automatically to whatever remains. Give history its own budget and it competes with retrieval, and one of them will lose unpredictably.

The rules that follow

Fixed claimants go first, in stable order. System prompt and tool definitions are the cache prefix. Anything that reorders or rewrites them costs you the cache on every call.

State is capped and regenerated, never accumulated. If the ledger exceeds its cap, that's a signal the run has too many items in flight — not a reason to raise the cap.

Retrieval gets a hard cap, and the cap is a feature. Ten thousand tokens of retrieved material forces a relevance decision. Uncapped retrieval returns its top N regardless of whether any of it is relevant, and the irrelevant portion is paid for on every subsequent turn.

The current turn is always reserved. Never let compaction eat the space the actual task needs.

What happens when a claimant overflows

Each needs an explicit policy, because the default — silently truncate whatever is at the end — is the worst option:

• Retrieval over cap → return fewer results with an explicit note that more matched. The agent can then narrow its query.
• A single tool result over cap → write it to a reference and return a handle plus a summary. This is the most common overflow and the easiest to fix.
• History over its residual → compact the oldest span, having already extracted anything worth keeping into state or notes.
• State over cap → escalate. Something is wrong with the run's structure.
• ✅ Constraints and the current task → never dropped, under any pressure. If the budget can't fit them, the run should fail rather than proceed without them.

🔍 Instrument the allocation

Log per-claimant token counts on every turn:

turn 14   sys 1,020 | tools 1,980 | state 1,340 | retr 9,800
          hist 15,200 | turn 890   | total 30,230 / 32,000

Three things become visible immediately: which claimant is growing, when compaction fired, and whether retrieval is using its cap or wasting it. A jump in any column is a defect with an address — and this is the single most useful instrument for long-run debugging, because it turns "it lost the plot" into "retrieval took 18k on turn 12."

💡 The anti-pattern

Filling the window because it's available. A larger window removes the forcing function that was producing better runs — when space was scarce, someone had to decide what mattered, and that discipline was worth more than the constraint cost.

Set the working budget at a level that forces the decision, independent of what the window permits. If a bigger model arrives with more room, that's headroom for the occasional large task, not a reason to raise every allocation.

The takeaway

Six claimants compete for one space, and unallocated space gets taken by whichever tool returned something large. Fix the shares: prompt and tools first for cache stability, state capped and regenerated, retrieval hard-capped so relevance is forced, the current turn always reserved, and history as the residual that compacts to fit. Then log the per-claimant counts each turn — that table is where long-run failures become visible before they become mysterious.