Why Turn 20 Costs More Than Turn 2

A first agent project usually produces a surprising bill, and the surprise comes from an assumption carried over from ordinary API work: that each call costs roughly the same. In an agent loop it doesn't, and the shape of the growth is worth understanding before you optimize anything.

Each turn re-sends everything

The model is stateless. It doesn't remember turn one when processing turn two — the loop resends the whole conversation every time.

So a six-turn run doesn't send the context once. It sends:

turn 1:  system + request                        ~2,000 tokens
turn 2:  + turn 1's reply + tool result          ~3,500
turn 3:  + turn 2's                              ~5,000
turn 4:  + turn 3's                              ~9,000   ← a big tool result landed
turn 5:  + turn 4's                             ~10,500
turn 6:  + turn 5's                             ~12,000
                                          total ~42,000 input tokens

Forty-two thousand input tokens for a conversation whose final state is twelve thousand. That gap is what an agent costs, and it's the fundamental fact of agent economics.

Two consequences

Something expensive early is paid for repeatedly. The four-thousand-token tool result at turn four is re-sent at turns five and six. Landing at turn two instead, in a twenty-turn run, it would be paid eighteen more times. → This is why moving large results out of context — a reference, a summary, a scratchpad note — saves so much more than its own size suggests.

Turn count matters more than it looks. Going from six turns to twelve doesn't double cost; it more than doubles it, because the later turns are the expensive ones. Anything that reduces turns — batch tools, parallel calls, better tool descriptions — is worth more than it appears.

Input, output, and cached are three different prices

Three counters, priced differently, and the intuition from chat usage points at the wrong one:

• Input tokens — everything sent. In agents this dominates, usually by a lot.
• Output tokens — what the model generates. Priced higher per token, but there are far fewer of them.
• Cached input tokens — a repeated prefix, priced substantially lower.

That third one is where the leverage is. If the beginning of your context is byte-identical between calls, it can be reused rather than reprocessed. In a loop that resends a growing array, an increasing share of each call is a prefix it already sent — exactly the case caching is for.

⚠️ And exactly the case a small mistake ruins. A timestamp in the system prompt, a tool list built from an unordered set, a summary regenerated each turn — any of these changes the prefix and the reuse is lost on every call. A cache that isn't hitting is a common cause of an agent costing several times what it should.

Where beginners lose money

No turn limit. A loop that can't tell it's finished runs until something stops it. Set a cap on the first day.

Tool results returning everything. An API's full response instead of the five fields the task needs — paid for on every later turn.

Retries that resend the context. A failed tool call retried at the model level costs a full turn, and a failure late in a run costs a large one.

Reflection and verification steps added by default. Each doubles the cost of the step it checks. Worth it against an oracle; not worth it as a general quality setting.

Long system prompts. Sent on every call in every run. A two-thousand-word prompt is a permanent tax, and the audit that removes half of it is usually available.

✅ The cheap wins, in order

1. Stabilize the prefix so caching works. Often the single largest saving, and it's a small fix.
2. Trim tool results to the fields the task uses.
3. Cap turns, and make finishing an explicit signal.
4. Batch tools so five calls become one — fewer turns and less context.
5. Move large results out of context, referenced by a handle.

The number to watch

Not total spend — cost per completed task, at p95. Total spend rises with usage, which is fine. Unit cost rising is a regression, and the p95 rather than the median because a tail of expensive runs is where agent money actually goes.

Track turns per task alongside it. Turns move first, so a rising turn count is the earliest warning that cost is about to follow.

The takeaway

An agent's cost is the sum of a growing array re-sent on every turn, not the size of the conversation. That makes early large results expensive, turn count superlinear, and prefix caching the highest-leverage optimization available. Cap the turns, trim what tools return, keep the prefix stable, and watch cost per completed task at p95 rather than the monthly total.