#latency
Every post tagged "latency".
14 posts · showing 11–14
-
Rate Limits Are an Agent Design Constraint, Not an Ops Problem
Backoff assumes a predictable call count. Agents burst, retry at three layers, and generate more traffic the more they're throttled — which makes rate limits an architecture question, not an ops ticket.
-
Streaming an Agent's Progress Without Lying About It
Token streaming goes quiet exactly when an agent gets slow — inside the tool call. The fix is an event stream from the loop, and the hard part is not inventing certainty the loop doesn't have.
-
The Triage Agent That Should Have Been a Filter
A routing agent replaced a set of rules and made the common case slower, pricier, and less predictable. The lesson isn't "don't use agents" — it's that judgment is worth paying for only where judgment is required.
-
The Retry Storm Your Dashboard Can't See: When Agent Failures Multiply Instead of Add
Three retry layers, each sensible on its own, compose by multiplication — and a failure at step fourteen costs the whole context, not the failed call. Your per-call dashboard will stay green the entire time.