#llm
Every post tagged "llm".
20 posts · showing 1–10
-
Three Ways to Get Valid JSON, and What Each One Costs
The retry loop was working and quietly biased: the tickets needing three attempts weren't a random sample, and their eventual valid output was thinner than the truth.
-
What Falling Inference Costs Change About Agent Design
Relaxed cost pressure mainly buys back the verification steps you declined. It doesn't rehabilitate filling the context or adding turns — those were quality problems wearing a cost disguise.
-
The Case for a Local-First Agent Stack
The cost argument for local-first is weak and the ideology one is worse. The case that holds is dependency control — and it comes with a permanent operational tax nobody budgets.
-
What Runs Locally Today and What Still Won't Soon
Local handles the steps; hosted handles the orchestration. The agent loop is the hardest thing to localize, because it stresses all three local constraints at once.
-
Will On-Device Intelligence Be Free? What the Economics Say
"Free" means three different things here. Two are plausible and already partly true; the third doesn't follow, because the frontier is defined by requiring the most compute.
-
How to Benchmark Models on Your Own Task
Public benchmarks tell you about a model, not about your system. Frozen tool responses, assertions written from the requirement, and four numbers instead of one — plus the shortcut when a full suite isn't happening.
-
When a Small Model Beats a Big One, and How to Tell
Sometimes "beats" means cheaper at acceptable quality — and sometimes it means genuinely better on the metric that matters. Four cases where smaller wins outright, and four where it looks like it should and doesn't.
-
The Cost of Choosing Your Model Too Early
The choice is rarely the mistake — the accumulated compensations are. Prompts absorb quirks, the harness papers over weaknesses, and none of it looks like coupling until you try to change something.
-
Matching Model Tier to Task: A Practical Table
"Simpler tasks get cheaper models" isn't actionable, because simplicity isn't the predictor. Two properties are — and one row of the table overrides all the others.
-
Should You Always Reach for the Flagship Model?
"Always use the cheapest model that works" has costs that don't appear in a token bill. The case for flagship-by-default is stronger than usually admitted — and there's a cheaper optimization people skip first.