#token-cost
Every post tagged "token-cost".
42 posts · showing 11–20
-
What Falling Inference Costs Change About Agent Design
Relaxed cost pressure mainly buys back the verification steps you declined. It doesn't rehabilitate filling the context or adding turns — those were quality problems wearing a cost disguise.
-
What Runs Locally Today and What Still Won't Soon
Local handles the steps; hosted handles the orchestration. The agent loop is the hardest thing to localize, because it stresses all three local constraints at once.
-
Will On-Device Intelligence Be Free? What the Economics Say
"Free" means three different things here. Two are plausible and already partly true; the third doesn't follow, because the frontier is defined by requiring the most compute.
-
How to Benchmark Models on Your Own Task
Public benchmarks tell you about a model, not about your system. Frozen tool responses, assertions written from the requirement, and four numbers instead of one — plus the shortcut when a full suite isn't happening.
-
When a Small Model Beats a Big One, and How to Tell
Sometimes "beats" means cheaper at acceptable quality — and sometimes it means genuinely better on the metric that matters. Four cases where smaller wins outright, and four where it looks like it should and doesn't.
-
Matching Model Tier to Task: A Practical Table
"Simpler tasks get cheaper models" isn't actionable, because simplicity isn't the predictor. Two properties are — and one row of the table overrides all the others.
-
Should You Always Reach for the Flagship Model?
"Always use the cheapest model that works" has costs that don't appear in a token bill. The case for flagship-by-default is stronger than usually admitted — and there's a cheaper optimization people skip first.
-
Longer Context Windows Won't Fix Long-Horizon Tasks
The premise is right — things fall out of context — and the conclusion doesn't follow. Most long-run failures aren't about storage, and a bigger window removes the discipline that was helping.
-
What Gets Cheap Next, and What Stays Expensive
Costs are being rearranged, not reduced. Four things are getting cheap, six aren't — and a fourth category nobody plans for is getting more expensive, including maintaining everything that was cheap to create.
-
Which Agent Alerts Should Wake Someone Up?
An agent can be green on every service metric while doing the wrong work at ten times the cost. One rule sorts the alerts — and the highest-value one is the one almost nobody has configured.