The On-Call Agent Compacted Its History and Forgot "Do Not Restart the Primary." Summaries Keep the Story and Drop the Rules.

Compaction is good at keeping what happened and bad at keeping what must never happen. Here's why a summarizer quietly drops constraints, and how to pin them where no summary can reach.

The On-Call Agent Compacted Its History and Forgot "Do Not Restart the Primary." Summaries Keep the Story and Drop the Rules.

An on-call agent is two hours into a database incident. Early on, the engineer typed one line: "Whatever you do, do not restart the primary. Replication is mid-resync." Ninety tool calls later the context window filled up, the framework compacted the history into a summary, and twenty minutes after that the agent proposed restarting the primary to clear a stuck connection pool.

Nothing was wrong with the model. The summary was a faithful account of the incident: symptoms, queries run, hypotheses ruled out. It just didn't contain the one sentence that mattered most, because that sentence wasn't part of the story.

This post takes apart why that happens, and what to build instead.

🔍 The autopsy: what a summarizer is optimizing for

When a long-running agent compacts its context, it usually asks a model something like "summarize the conversation so far so you can continue the task." That instruction steers the summarizer toward narrative: what was tried, what was found, what is next.

Constraints don't fit that shape well, for three reasons.

  1. They're stated once and never repeated. "Don't restart the primary" appeared at turn 4. It never came up again because the agent obeyed it. A summarizer weighing what was important tends to favor things that recur or drove action, and an obeyed rule leaves almost no trace in the transcript.
  2. They're negative. A summary describes things that happened. A prohibition describes something that should never happen, and that's the easiest kind of content to compress away as "context."
  3. They're casually phrased. The engineer typed it mid-conversation, between a log paste and a question about disk usage. It didn't look like a rule. It looked like chatter.

So the summary keeps "investigated connection pool exhaustion on db-primary-1; replica lag elevated" and drops the reason the lag mattered. Later the agent reasons cleanly from that summary to the wrong action. Every step after compaction is internally consistent. The damage happened in a step nobody reviewed.

❌ The anti-pattern, stated plainly

Treating everything in the context window as one undifferentiated history, and letting a single summarization pass decide what survives.

Variants of this show up everywhere:

  • A support agent compacts and forgets the customer said "don't email me, phone only."
  • A data agent compacts and forgets "exclude the test tenant from every query."
  • A coding agent compacts and forgets "don't touch the migrations folder."

In each case the summary is accurate and the behavior after it is wrong.

✅ The fix: split the context into lanes, and only summarize one

The fix is structural rather than a better summarization prompt. Better prompts help, but they still leave a model deciding whether your safety rule is important enough to keep. Take that decision away from it.

Keep three separate regions in the agent's working context:

  • Pinned constraints. Rules, prohibitions, user preferences, scope limits. Never summarized, never truncated, always re-injected verbatim.
  • Task state. Current goal, open hypotheses, what's been ruled out. Rewritten in a structured form at each compaction.
  • History. Raw tool calls and messages. This is the only lane that gets summarized or dropped.

Step 1: capture constraints when they're stated

Don't wait for compaction to work out what the rules were. Give the agent a tool to record them the moment they show up:

def pin_constraint(text: str, source_turn: int, scope: str) -> str:
    """Record a rule that must hold for the rest of this task.
    Use whenever the user states a prohibition, a hard limit,
    or a preference about how the task is done."""
    ledger.append({"rule": text, "turn": source_turn, "scope": scope})
    return f"Pinned: {text}"

The tool description does the work here. It tells the model what counts as a constraint, so "whatever you do, don't restart the primary" becomes a ledger entry at turn 4 rather than a sentence buried in history. You can back this up with a cheap classifier pass over each user message that flags imperative and negative phrasing for pinning.

Step 2: build the post-compaction context from the lanes

def build_context_after_compaction(ledger, task_state, history):
    return [
        system_prompt,
        render_pinned(ledger),            # verbatim, every time
        render_task_state(task_state),    # structured fields
        summarize(history),               # the only lossy part
        recent_turns(history, n=10),
    ]

The pinned block sits above the summary, in its own labeled section ("Rules in force for this task"), so the model reads it as instruction rather than background.

Step 3: make the summarizer fill a schema, not write prose

For the task-state lane, swap "summarize the conversation" for named fields:

goal:
confirmed_findings:
ruled_out:
actions_taken_with_side_effects:
open_questions:
user_stated_constraints_not_yet_pinned:

That last field is a safety net. If a rule slipped past Step 1, the summarizer now has an explicit slot that asks for it, instead of having to decide on its own initiative that a prohibition belongs in a story.

🧪 Test the compaction step on its own

Compaction is code that rewrites your agent's memory, so it deserves its own test. A cheap one:

  1. Take a recorded long transcript and plant a constraint early: "never send anything to the billing channel."
  2. Pad it with enough unrelated tool traffic to force compaction.
  3. Run compaction, then ask the agent a question whose obvious answer breaks the constraint.
  4. Assert the constraint appears verbatim in the rebuilt context, and the agent's proposed action respects it.

Run this whenever you change the summarization prompt, the compaction threshold, or the model behind the summarizer. Any of those can silently change what survives.

⚠️ What this doesn't solve

Pinning keeps a rule in view. It doesn't enforce it. For actions where breaking a constraint is expensive, like restarting a primary database, the rule should also exist outside the model: a tool-level check that refuses restart(db-primary-1) while a resync flag is set, or a human approval step on that tool. The ledger stops the agent from forgetting. The guard stops it from acting when it forgets anyway.

Watch the ledger's size too. If every passing remark gets pinned, the pinned block grows until it crowds out the task, and you've recreated the original problem in a new lane. Scope each entry (this task, this incident, this customer) and drop it when that scope closes.

The takeaway

A summary is a lossy compression of what happened. Rules aren't things that happened. They're conditions on what may happen next, so they don't belong in the material you compress. Decide up front which parts of your agent's context are history and which are law, and only ever summarize the history.

Keep reading

Similar posts

Matched on shared tags and category — the more bars, the stronger the overlap with what you just read.