#token-cost
Every post tagged "token-cost".
42 posts · showing 1–10
-
Your Inbox Agent Wakes Every 15 Minutes to Find Nothing. Cron or Event Trigger? Decide by What a Wasted Run Costs.
The trigger is the least-discussed design choice in a personal agent, and it quietly decides your bill, your duplicate rate and how stale its answers are. A simple test for choosing between a schedule and an event, and why the best setup usually uses both.
-
Your Coding Agent Hit Its Cost Cap Mid-Refactor and Left Nothing Behind. Reserve the Last Fifth of the Budget for the Handoff.
A hard spend cap protects your bill, but when it trips mid-task it throws away everything the agent learned. Split the budget into phases so the run always ends with a usable handoff instead of a dead process.
-
An Agent That Cannot Fail Will Retry Forever. Budget Tokens and Wall-Clock, Not Retry Counts.
Your max_retries=3 never fires, because every search call succeeds. The agent just keeps finding new ways to look for an answer that doesn't exist. Here is the budget that actually stops it, and the exit that lets it stop politely.
-
Draining a Data Backlog: Should the Agent Batch Rows or Take Them One at a Time?
The ratio of preamble tokens to row tokens decides whether chunking a backlog saves an order of magnitude or nothing at all — and it's the alignment, retry amplification and blast radius that decide whether you should anyway.
-
Retry, Reroute, or Hand Back: Budgeting an Agent's Second Chances
A tool call fails and the loop tries again — five times, forty seconds, and still nothing usable for the user. Retrying is only one of three legal responses, and picking the wrong one turns a fast failure into an expensive one.
-
The Supervisor That Should Have Been a For-Loop
Every night the same plan, a model call per dispatch, and about once a week it checked eleven services instead of twelve. A for-loop cannot skip an item; an agent orchestrating one eventually will.
-
Handing an Agent a PDF: Three Paths Into the Context
A PDF never enters a context window — something derived from it does. The counterparty name right and the date wrong usually means the document took the wrong path in.
-
Your Trace Store Will Outgrow Your Model Bill
Storage passed inference cost in three weeks, and the proposed fix — sample at 5% — would have discarded exactly the runs worth keeping. A trace isn't one artifact with one lifetime.
-
Budgeting the Context Window: Dividing a Fixed Space
The window wasn't full and the constraints still vanished — because six claimants compete for one space and nobody allocated it. Making history the residual rather than a claimant is what makes the scheme work.
-
The Economics of an Agent-Heavy Team
Several line items move and only one of them is tokens. The savings are real, land on a single term, and are partly consumed by four others that grow — which is why big speedups produce modest delivery gains.