#reliability
Every post tagged "reliability".
124 posts · showing 91–100
-
Getting an Agent to Say How Sure It Is
Ask a model how confident it is and you get a plausible number that doesn't move with difficulty. Five signals that actually correlate with being right — and the calibration check that tells you if yours is noise.
-
Few-Shot Examples: When They Help an Agent and When They Hurt
Two or three similar examples is the worst case — enough to teach a template, not enough to convey the range. And the example nobody includes is the one where the right answer is to decline.
-
Bisecting a Bad Run: Finding the Change That Broke It
It worked last week. Six things could have changed, and they vary independently — so bisect across the dimensions first, then within one. And diff the raw arrays for what's missing, not what's different.
-
State Machines for Agents in Regulated Flows
When doing steps out of order is a compliance failure regardless of outcome, a model-decided control flow is the wrong shape. Put the ordering in code, the conversation in the agent, and scope the tools per state.
-
The Migration Agent: 4,000 Files and What Made It Tractable
The difference between a migration agent that saves weeks and one that produces four thousand unreviewable diffs is decided before it touches a file — and it starts with building the checker, not the agent.
-
Testing Multi-Agent Systems Without Losing Your Mind
Add a second agent and end-to-end assertions stop telling you anything useful. Contracts at every seam, stubs for the supervisor's unhappy paths, and integration tests that assert on graph shape and cost.
-
Deadlock, Livelock, and the Agent That Waits Forever
A deadlocked thread pool gives you a stack trace. Deadlocked agents produce fluent, plausible activity that looks like work — and bill you for it until someone notices.
-
The Verifier in the Loop: Designing an Oracle Your Agent Can Iterate Against
Coding agents work as well as they do because compilers exist. In other domains the oracle is something you build — and most tasks have one hiding in them if you spend ten minutes looking.
-
What a System Prompt Should and Shouldn't Contain
Five things belong in a system prompt. Most of what accumulates there — tool instructions, deterministic rules, incident patches, data — belongs somewhere it would actually work.
-
Queueing and Backpressure for Agent Workloads
A run can take four seconds or four minutes, and you can't tell in advance. Size the pool to the provider's rate limit, split queues by shape — and check the one timeout relationship that silently runs your task twice.