Your Trace Store Will Outgrow Your Model Bill

A support team turned on full trace capture, correctly, because debugging agents without the raw arrays is guesswork. A few weeks later the trace storage line was larger than the inference line, and someone proposed sampling at 5% — which would have thrown away exactly the runs worth keeping.

Both the cost and the proposed fix come from the same misunderstanding: treating a trace as one thing with one retention policy.

Why agent traces are unusually heavy

An agent re-sends its accumulated context on every turn. Capture each call naively and you store that growing context once per turn.

A twenty-turn run whose context reaches, say, 30k tokens doesn't store 30k tokens of payload. It stores something closer to the sum of every intermediate array — the same system prompt twenty times, the same tool definitions twenty times, the early tool results nineteen more times after they first appeared.

→ The storage cost of a run scales roughly the way its token cost does, which is why the trace bill tracks the model bill and then passes it: you pay the provider for input tokens once per call, and you pay storage for those same bytes forever.

The dedupe that removes most of it

Content-address the payloads. Hash each message; store each distinct message once; have the per-turn record reference hashes rather than embed text.

turn 14 payload:  [h:9f2c..., h:e7c3..., h:41ab..., h:88de...]
                   system    tools     doc_44    turn-14 task

The system prompt is one stored object referenced by every turn of every run. Tool definitions likewise. A tool result that persists across sixteen turns is stored once.

⚠️ This works because agent contexts are built for prefix caching — the stable prefix that makes caching effective is the same property that makes deduplication effective. Teams that have already stabilized their prefix for cost reasons get a large dedupe ratio for free. Teams whose prefix churns get neither benefit, which is one more reason to fix it.

Three tiers, three retention policies

The mistake is one policy for the whole trace. Split it:

Tier 1 — the run record. Outcome, turn count, token counts, cost, latency, tool sequence, model and prompt versions, what the user did next. A few hundred bytes.

Keep forever. This is what answers "was last quarter better than this one," and it's small enough that retention is never the reason to drop it.

Tier 2 — the span structure. Per-step timings, tool names, arguments (hashed or truncated), result sizes, cached vs uncached tokens. Kilobytes.

Keep months. This is what diagnoses a latency or cost regression, and what makes tool-sequence analysis possible.

Tier 3 — the payloads. Rendered arrays and full tool results. Megabytes, and effectively all of the storage.

Keep days to weeks — with exemptions. This is what you need to debug one specific run, and you overwhelmingly need it for runs that went wrong, recently.

The exemption rule is the whole design

Uniform sampling at 5% is the wrong instrument, because the interesting runs are rare by definition. Random sampling keeps 5% of the boring ones and discards 95% of the failures.

Retain payloads in full, permanently or long-term, for any run that:

• errored, escalated, or halted
• exhausted its turn or token budget
• cost or ran more than some multiple of the median
• took an unusual tool sequence
• the user rephrased or abandoned after
• a human flagged in review

✅ Everything else gets short retention plus a small random sample for baseline comparison. That inverts the naive policy: you keep almost all of the anomalies and almost none of the routine, which is the opposite of what percentage sampling does and the same order-of-magnitude saving.

⚠️ Payloads are also where the personal data is

Cost is the visible problem. Retention of tier 3 is also a compliance question, because the rendered arrays contain whatever the agent read — support tickets, customer records, documents.

Two consequences:

Redact at capture, not at query. A pipeline that redacts on read has already stored the raw data, and the retention clock has already started on the unredacted copy.

Tier 3 retention has a legal ceiling, not just a budget. If your policy says customer data is deleted after N days, a trace store holding it for a year is a finding regardless of how useful it is. This is the constraint that should set the number, with cost as the secondary input.

🔍 What you lose by cutting each tier

Worth knowing before someone proposes a cut under budget pressure:

• Cut tier 1 → you can no longer say whether anything is getting better or worse. Never cut this; it's the cheapest and most valuable data you have.
• Cut tier 2 → you keep the aggregate view and lose the ability to explain it. A cost regression becomes visible but not diagnosable.
• Cut tier 3 → you keep the ability to spot a bad run and lose the ability to understand it. Survivable if the exemptions are right, which is why the exemption list matters more than the retention window.

The takeaway

A trace isn't one artifact with one lifetime. Content-address the payloads so a stable prefix is stored once rather than once per turn, keep the run record forever, keep the span structure for months, and keep payloads briefly — with the anomalous runs exempt from expiry entirely. Redact at capture, and let your data policy rather than your budget set the tier-3 ceiling. Percentage sampling looks like the obvious lever and it discards precisely the runs you turned tracing on to see.