#reliability
Every post tagged "reliability".
124 posts · showing 1–10
-
The Analytics Agent Answered "What Was Q3 Revenue?" From a Deprecated Table. Make It Name Its Source Before Its Number.
The SQL was valid, the arithmetic was right, and the number was still wrong, because the agent picked the table whose name looked most like the question. Here is how to make a data agent commit to a source you can check before it hands anyone a figure.
-
The Invoice Tool Timed Out, So the Billing Agent Called It Again. The Customer Was Charged Twice. A Timeout Is Not a Failure.
A timeout tells the agent it stopped waiting, not that nothing happened. Here is how a reasonable-looking retry turned into a double charge, and the idempotency key that makes the retry safe.
-
Your Inbox Agent Wakes Every 15 Minutes to Find Nothing. Cron or Event Trigger? Decide by What a Wasted Run Costs.
The trigger is the least-discussed design choice in a personal agent, and it quietly decides your bill, your duplicate rate and how stale its answers are. A simple test for choosing between a schedule and an event, and why the best setup usually uses both.
-
Your Research Agent Scored 100% on Citations. The Grader Only Checked That a Source Was Attached.
An eval that has never failed anything is not proof of a good agent. Before you trust a grader's score, feed it answers you know are wrong and see if it notices.
-
The Travel Agent Reviewed Its Itinerary Three Times and Approved the Wrong Date Every Time. Reflection Needs Something to Check Against.
A self-review loop that reads only its own draft can polish the wrong answer forever. The fix is to give the critic something the generator never had: the original request, a fresh context, or a tool that can say no.
-
The Expense Agent Logged "Rate limit exceeded" as a Vendor Name. Make Tool Errors Impossible to Mistake for Results.
When a tool returns its failure as a plain string, the model reads it like any other output and carries on. The fix lives in the tool's return shape, not in the prompt.
-
The Analytics Agent's SQL Was Correct. It Answered a Different Question. Make It Print Its Definitions Before Its Number.
The query ran clean, the chart looked right, and the churn number went into the board deck. It counted trial accounts as customers. The fix is not better SQL; it is making the agent commit to what every word in the question means before it touches the warehouse.
-
Your Scheduling Agent Moved One Meeting and Said "Done." Teach the Loop What Finished Looks Like.
An agent loop ends when the model stops asking for tools, not when your task is finished. Here is why that gap produces confident half-done work, and a small done-check that closes it.
-
Your Research Agent Passed the Eval Once. Run Every Case Five Times and Grade the Worst Run.
A single green run tells you the agent can do the task, not that it will. Repeat each eval case, split the suite into solid, flaky and broken, and the flaky bucket becomes your real to-do list.
-
The Inbox Agent Said Nobody Had Replied. The Search Tool Had Returned Page One of Four.
An agent can't tell "there's nothing here" from "I only saw the first page" unless the tool tells it. Here is how to make a paginated tool announce its own incompleteness, and why claims of absence need a stricter rule than claims of presence.