#token-cost
Every post tagged "token-cost".
22 posts · showing 11–20
-
Migrating Model Versions Without Breaking Your Agent
The new model is better at nearly everything and different at everything, and tool selection is what shifts first. Replay a frozen eval set against both — and put the cost-per-task row in the comparison.
-
Model Routing: Using the Cheap Model for the Cheap Steps
Classification, extraction, and formatting move to a cheaper tier safely. Tool selection and planning don't — and the failure looks like a tool-description bug, so you'll debug the wrong thing for a week.
-
Cost Attribution: Finding Which Feature Is Eating Your Token Budget
Agent spend is dominated by a tail of expensive runs, so every averaged chart hides the cause. Five tags on each model call, one table of your twenty priciest runs, and the mystery usually resolves in an afternoon.
-
Specialist Agents or Specialist Prompts? The Cheaper Half of Multi-Agent
A separate agent buys separate context, separate permissions, and separate lifecycles. It does not buy expertise — that's what prompts are for, and they cost nothing and lose nothing at the boundary.
-
Where You Put Things in the Context Changes the Answer
Same instructions, same documents, different results — because the constraint sat in the middle of a long array. Two ordering rules, one of which also decides whether your prompt cache ever hits.
-
Chatbot, Workflow, or Agent? Pick the Simplest Thing That Works
Six weeks building an agent for a task with three fixed steps. One question — can you draw the flowchart before seeing the input? — separates the three architectures and saves most of that time.
-
Debate or Vote? Getting Real Value From Multiple Model Opinions
Voting harvests independent error; debate harvests challenge. Pick wrong and you multiply your token bill for agreement you'd have had from a single call — and the disagreement, the part worth paying for, gets logged away.
-
Rate Limits Are an Agent Design Constraint, Not an Ops Problem
Backoff assumes a predictable call count. Agents burst, retry at three layers, and generate more traffic the more they're throttled — which makes rate limits an architecture question, not an ops ticket.
-
Chain, Blackboard, or Broadcast? Three Ways Agents Share State
Chains lose the evidence, blackboards fight over it, broadcasts drown in it. Pick by which failure your task can absorb — and log provenance from day one or debugging becomes guesswork.
-
The Triage Agent That Should Have Been a Filter
A routing agent replaced a set of rules and made the common case slower, pricier, and less predictable. The lesson isn't "don't use agents" — it's that judgment is worth paying for only where judgment is required.