Running in Production
Deployment, cost, latency, monitoring, scaling.
24 posts · showing 1–10
-
Your Inbox Agent Wakes Every 15 Minutes to Find Nothing. Cron or Event Trigger? Decide by What a Wasted Run Costs.
The trigger is the least-discussed design choice in a personal agent, and it quietly decides your bill, your duplicate rate and how stale its answers are. A simple test for choosing between a schedule and an event, and why the best setup usually uses both.
-
Your Coding Agent Hit Its Cost Cap Mid-Refactor and Left Nothing Behind. Reserve the Last Fifth of the Budget for the Handoff.
A hard spend cap protects your bill, but when it trips mid-task it throws away everything the agent learned. Split the budget into phases so the run always ends with a usable handoff instead of a dead process.
-
Your Agent Fleet Kept Its Own Quota Pinned for Forty Minutes: Synchronised Retries Are a Self-Inflicted Outage
The provider rejected one batch of calls; the fleet turned that into forty minutes of self-inflicted downtime. Why exponential backoff alone makes synchronised retries worse, the sub-second signal that exposes it, and the three-part fix that actually breaks the loop.
-
Your Agent Works in the Demo Because You're Watching: Six Failures That Only Appear at 3 A.M.
A live demo isn't a smaller version of production — it's a different environment with a human silently patching over every gap. Here are the six failure classes that only wake up once that human goes to bed.
-
A 5% Canary Mostly Samples Your Easy Tasks. Roll Agent Upgrades Out by Task Class.
Random traffic slices sample volume, but an agent's risk lives in the thin tail of long, tool-heavy, irreversible tasks. Here's how to build a rollout ladder that samples the danger instead.
-
One 300-Second Timeout Tells You Nothing. Agents Need a Deadline Budget.
A run that dies at its ceiling with no attribution tells you nothing about which part was slow — and pushes you toward the laziest fix, raising the ceiling. Three nested clocks, an inherited remainder, and a reserved tail slice turn timeouts into a signal you can actually read.
-
One Worker Pool or Two? Separating Fast Agent Runs From Slow Ones
A handful of long research runs can starve hundreds of eight-second ticket replies without a single error firing. The fix isn't more workers — it's deciding, at admission, which lane a run belongs in.
-
Your Agents Asked 40 Questions This Week. Which Ones Should Have Reached a Human?
Human attention is the one resource in an agent deployment that never shows up on the bill — so agents spend it freely. Here's a three-way routing rule, a threshold you can actually tune, and the two metrics that tell you when your agent is asking too much or too little.
-
Retry, Reroute, or Hand Back: Budgeting an Agent's Second Chances
A tool call fails and the loop tries again — five times, forty seconds, and still nothing usable for the user. Retrying is only one of three legal responses, and picking the wrong one turns a fast failure into an expensive one.
-
The First Hour of an Agent Incident
Nothing crashed, no alert fired, and a customer just forwarded a policy answer your support agent invented. The hardest question in the first hour isn't what broke — it's when it started.