#failure-analysis
Every post tagged "failure-analysis".
8 posts · showing 1–8
-
When Memory Makes an Agent Worse
Every retrieved memory is content injected into the context, relevant or not. Three ways that degrades the agent — including the one where it argues with a user about their own stated preference.
-
Deadlock, Livelock, and the Agent That Waits Forever
A deadlocked thread pool gives you a stack trace. Deadlocked agents produce fluent, plausible activity that looks like work — and bill you for it until someone notices.
-
The Agent That Read Page 1 of 40: Pagination in Tool Design
It reported three overdue invoices. There were sixty-one. The tool worked as documented — which is exactly why this bug is so common and so quiet.
-
What Breaks First When You Give an Agent a Browser
Six failures, in the order they arrive: the page isn't what you think it is, selectors rot, state assumptions break silently, and the session carries more authority than the task needs.
-
The LLM Judge That Agreed With Everything
A suite scored 94% and 93% on two versions — but the judge would have passed almost anything. Four biases quietly manufacture those numbers, and one prompt change fixes more than the rest combined.
-
The Planner That Plans Too Much: When Upfront Decomposition Hurts
An agent found the root cause in step one and executed the remaining seven steps anyway. Plans in context outweigh the evidence that should kill them — unless revision is something the loop can actually do.
-
Why Your Agent Stops Too Early: The Missing Termination Signal
"Done" in most agent loops means the model stopped asking for tools — indistinguishable from stuck, confused, or bored. Making completion an explicit, validated claim takes about twenty lines.
-
The Retry Storm Your Dashboard Can't See: When Agent Failures Multiply Instead of Add
Three retry layers, each sensible on its own, compose by multiplication — and a failure at step fourteen costs the whole context, not the failed call. Your per-call dashboard will stay green the entire time.