#reliability
Every post tagged "reliability".
124 posts · showing 81–90
-
Designing the First Turn: What Happens Before Any Tool Call
Everything after it inherits the frame it sets. Four things belong in turn one — and when a run fails at turn twelve, this is usually where the defect was introduced.
-
The QA Agent That Wrote Tests That Always Passed
Coverage climbed, the build stayed green, and a real regression sailed through. An agent asked for passing tests writes passing tests — and the easiest passing test verifies nothing.
-
Eval Cases Where the Right Answer Is "I Don't Know"
Suites measure whether the agent got it right, never whether it should have answered at all. Five case types where the correct response is a question — plus the paired cases that stop you from just making it timid.
-
Retry Semantics: Which Tools Are Safe to Call Twice
Agents repeat tool calls for at least five ordinary reasons — none of them bugs. Whether that's harmless or a duplicate charge is a property of the tool, and usually one nobody wrote down.
-
Graceful Degradation When the Model Provider Is Down
A fallback exercised only during incidents is exercised for the first time during an incident. Six degradation rungs, and the three cheapest ones cover most of the value.
-
Request, Worker, or Durable Execution? Hosting Models for Agents
Where a run lives decides how long it can be, whether it survives a deploy, and whether it can wait for a person. Pick by the longest wait it can hit — and keep state out of process memory so the answer can change.
-
The Sales-Research Agent and the Problem of Stale Facts
Every fact in the brief had been true. None was labelled with when — so someone opened a call congratulating a person on a role they'd left a year ago.
-
When Memory Makes an Agent Worse
Every retrieved memory is content injected into the context, relevant or not. Three ways that degrades the agent — including the one where it argues with a user about their own stated preference.
-
Shared Tool Access: When Two Agents Touch the Same Resource
An agent holds a stale read for seconds, not microseconds — long enough that last-write-wins silently deletes another agent's work while both report success. Partition, append, or version.
-
Designing Tools for Agents vs Designing APIs for Developers
One endpoint, one tool is the obvious mapping and the wrong one. APIs assume a consumer who reads docs once and writes a loop — an agent re-decides everything from a paragraph, every call.