#deep-dive
Every post tagged "deep-dive".
107 posts · showing 1–10
-
The Agent Got the Right Answer by the Wrong Route. Score the Trajectory, and Know the Three Cases Where the Answer Is Enough
A final-answer eval passes an agent that guessed, one that ran a write query mid-task, and one that took forty steps to do the job in four. Here is how to assert on the path without freezing a golden trace, and when you can safely skip it.
-
A Second Pass That Asks the Same Question Finds the Same Bugs. Change the Unit, Not the Temperature.
Raising the temperature on a review pass re-rolls the dice inside a frame that has already decided what is findable. Three properties determine what a pass can see, and a real second pass has to change at least two of them.
-
Forty Tools in One Prompt: What It Actually Costs, and the Three Signals That Say Split the Agent
Forty registered tools do not fail loudly — the agent just gets vague, picking near-synonyms and asking questions it had the answer to. Here are the three costs you're paying, and the signals in your traces that tell you to merge, gate, or finally split.
-
A Two-Line Prompt Diff Is a Behavior Change. Here's the Regression Check It Needs.
A prompt edit has the blast radius of a code change and none of the safety rails — no compiler, no type error, no failing test. Six checks that turn a forty-second approval into a real regression gate.
-
Your Agent's Dry Run Doesn't Bind the Real Run — Confirm the Plan Hash, Not the Intent
A human approves a preview; the agent then executes a freshly computed call. Everything that shifts in between is unguarded — and the fix is a redeemable plan object, not a sterner confirmation prompt.
-
Your Tool Returned 4,000 Rows and the Agent Needed Three: Truncate, Paginate, or Summarize?
Most tool endpoints were written for a caller that can loop and discard. An agent reads the whole response in one gulp and keeps it forever — and the three ways of shrinking that response fail in completely different ways.
-
One 300-Second Timeout Tells You Nothing. Agents Need a Deadline Budget.
A run that dies at its ceiling with no attribution tells you nothing about which part was slow — and pushes you toward the laziest fix, raising the ceiling. Three nested clocks, an inherited remainder, and a reserved tail slice turn timeouts into a signal you can actually read.
-
Untrusted Text Doesn't Stay Untrusted: Where Provenance Gets Lost in an Agent Pipeline
An injection scanner that fires on the raw email won't fire on the tidy summary of it. Untrusted content gets laundered clean every time an agent rewrites it — here's where the taint disappears, and how to make it survive.
-
Append or Supersede? What to Do When New Information Contradicts Agent Memory
Most agent memory systems only know how to add. The moment a fact changes, the store holds both versions and the retriever picks whichever one scores higher — which is why a scheduling agent keeps booking a room that was decommissioned last month.
-
Cached or Re-Called? Giving Every Agent Tool a Staleness Budget
An agent that reads a value once and acts on it four minutes later isn't reasoning badly — nobody told it how long that value stays true. Per-tool staleness budgets, and the one rule that matters most before a write.