Cost Attribution: Finding Which Feature Is Eating Your Token Budget

The monthly model bill went up by half. The provider dashboard shows total tokens by day and by API key. Both keys are named production. Nobody can say which feature caused it, which means nobody can decide whether it was worth it.

Cost attribution for agents is unusual because spend isn't proportional to requests. One user's forty-turn run can cost more than a thousand ordinary interactions, so aggregate metrics hide everything that matters. The fix is small and needs to be in place before you need it.

Tag at the call, not at the boundary

Every model call should carry the dimensions you'll want to slice by. At minimum:

{ run_id, user_id, tenant_id, feature, step, model, prompt_version,
  input_tokens, output_tokens, cached_tokens, latency_ms }

Three of these do most of the work and are most often missing:

• step — which part of the agent this call was. Planning, tool-selection, summarization, reflection, final synthesis. Without it, you know a run cost a lot but not which stage.
• cached_tokens — cached and uncached input tokens are priced very differently, and a cache-hit-rate regression is one of the most common causes of a sudden bill increase with no behavior change.
• prompt_version — so a cost change can be attributed to a specific prompt edit. This turns a mystery into a diff.

Aggregate per completed task

The unit that means something is the task, not the call. Roll up to:

• Cost per completed task, at p50 and p95. The p95 is the number that governs your bill, because the tail dominates.
• Cost per task by feature. Which product surfaces are expensive.
• Turns per task. Cost is mostly a function of turns, so this is the leading indicator that moves before the bill does.
• Cache hit rate. Should be high and stable for a well-structured agent; a drop is nearly always a context-ordering change.

→ The p95-to-p50 ratio is the single most diagnostic number here. A wide spread means a subset of runs behaves completely differently from the median, and those runs are usually where both your cost and your quality problems live.

Where the money actually goes

Four patterns account for most surprise costs, and tagged data identifies which one you have in minutes:

Context re-sending. Turn N re-sends everything from turns 1..N-1. A run with a big early tool result pays for it on every subsequent turn. Visible as input tokens growing superlinearly with turn count.

Cache misses from unstable prefixes. A timestamp in the system prompt, or a reordered tool list, and every call reprocesses from scratch. Visible as a low cached_tokens ratio, and it's usually a one-line fix.

Retry amplification. Failed calls retried at multiple layers, each re-sending the full context. Visible as high cost on runs whose task looks small.

Silent verbosity. Longer outputs on every call after a prompt change that encouraged thoroughness. Visible as output tokens per step rising with no change in turn count.

💡 The report worth building first

Before any dashboard, build one table: the twenty most expensive runs last week, with their tags and turn counts. Read them.

This finds more than any aggregate view, because expensive runs are usually expensive for a specific, fixable reason — a loop that didn't terminate, a tool returning an enormous payload, a retry storm. Aggregate charts average those away; the top-twenty list puts them in front of you.

Making the numbers actionable

✅ Budget per feature, alert on the derivative. Alert when cost per completed task rises by a threshold week over week, not when total spend crosses a line. Total spend rising with usage is fine; unit cost rising is a regression.

✅ Put unit cost in the eval report. If your eval suite reports cost per case alongside pass rates, a prompt change that improves quality by a point and doubles cost gets caught in review rather than in the invoice.

⚠️ Attribute at the tenant level if you're multi-tenant, even before anyone asks. The question "which customer is unprofitable?" arrives eventually, and retrofitting the tagging means losing the history.

The takeaway

Agent costs are dominated by a tail of expensive runs, so anything averaged hides the cause. Tag every model call with run, feature, step, prompt version, and cached-token counts; aggregate per completed task; watch p95 and cache hit rate rather than totals. Then read your twenty most expensive runs — the answer is usually sitting in three of them.