#deployment
Every post tagged "deployment".
19 posts · showing 11–19
-
Queueing and Backpressure for Agent Workloads
A run can take four seconds or four minutes, and you can't tell in advance. Size the pool to the provider's rate limit, split queues by shape — and check the one timeout relationship that silently runs your task twice.
-
Canary Releases for Prompt Changes
A sentence added to fix one behavior routinely changes three others. Ship prompts like behavior: versioned by rendered hash, split by stable key, and compared on tool distribution — the row nobody collects.
-
Agents That Wait: Callbacks, Webhooks, and Long-Running Tools
A twenty-minute export breaks the assumption that tools return quickly. Handles, in-loop polling, and true suspension — plus the expiry field that stops suspended runs leaking forever.
-
Migrating Model Versions Without Breaking Your Agent
The new model is better at nearly everything and different at everything, and tool selection is what shifts first. Replay a frozen eval set against both — and put the cost-per-task row in the comparison.
-
Designing a Kill Switch That Works Mid-Run
Killing the process leaves side effects in an unknown state. A stop that works checks between steps — before every model call and before every tool call — and the difference matters most at 2am.
-
Cost Attribution: Finding Which Feature Is Eating Your Token Budget
Agent spend is dominated by a tail of expensive runs, so every averaged chart hides the cause. Five tags on each model call, one table of your twenty priciest runs, and the mystery usually resolves in an afternoon.
-
Designing for Resumability: Agents That Survive a Restart
Serializing the message array is the obvious approach and it produces subtly wrong resumptions. What has to survive a crash is a work ledger, idempotent side effects, and enough state to rebuild context from scratch.
-
Rate Limits Are an Agent Design Constraint, Not an Ops Problem
Backoff assumes a predictable call count. Agents burst, retry at three layers, and generate more traffic the more they're throttled — which makes rate limits an architecture question, not an ops ticket.
-
Streaming an Agent's Progress Without Lying About It
Token streaming goes quiet exactly when an agent gets slow — inside the tool call. The fix is an event stream from the loop, and the hard part is not inventing certainty the loop doesn't have.