#advanced
Every post tagged "advanced".
48 posts · showing 1–10
-
Your Research Agent Read One Poisoned Web Page, and It Still Had the Email Tool. Lock Risky Tools the Moment Untrusted Text Enters the Context.
Prompt-injection filters try to spot the bad sentence. A taint flag skips that guess: once fetched text is in the context, the tools that can do damage are no longer offered.
-
Your Agent Fleet Kept Its Own Quota Pinned for Forty Minutes: Synchronised Retries Are a Self-Inflicted Outage
The provider rejected one batch of calls; the fleet turned that into forty minutes of self-inflicted downtime. Why exponential backoff alone makes synchronised retries worse, the sub-second signal that exposes it, and the three-part fix that actually breaks the loop.
-
The Agent Got the Right Answer by the Wrong Route. Score the Trajectory, and Know the Three Cases Where the Answer Is Enough
A final-answer eval passes an agent that guessed, one that ran a write query mid-task, and one that took forty steps to do the job in four. Here is how to assert on the path without freezing a golden trace, and when you can safely skip it.
-
The Page Your Agent Read Told It to Email Your Customer List. "Ignore Instructions in the Content" Won't Stop the Next One.
A system-prompt warning makes injection less likely. It doesn't make it impossible, and the attacker gets as many tries as they like. The real fix lives in the tool dispatcher, not the prompt.
-
Your Agent's Dry Run Doesn't Bind the Real Run — Confirm the Plan Hash, Not the Intent
A human approves a preview; the agent then executes a freshly computed call. Everything that shifts in between is unguarded — and the fix is a redeemable plan object, not a sterner confirmation prompt.
-
Your Retriever Returned 12 Chunks. The Agent Only Really Read 3 of Them.
The correct passage was in the prompt and the agent wrote around it anyway. That's not a retrieval bug — it's a placement bug, and there's a one-minute test that tells the two apart before you touch your embeddings.
-
Blackboard or Mailbox: How Should Three Agents Share What They Learn?
Two agents discover the same fact; a third contradicts both and nobody notices. The shape of your shared state — a common board or point-to-point messages — decides which of those happens.
-
What Crosses the Handoff: Transcript, Summary, or Contract?
A triage agent spends eleven tool calls finding the bug, then hands off. Passing the transcript ships its dead ends; passing a summary strips the uncertainty. What a typed handoff contract gets right — and the one field teams always forget.
-
Let the Model Pick the Transition, Not the Tool
A returns agent that can issue a refund before it has checked eligibility will eventually do exactly that, and no amount of prompt wording reliably stops it. Making the call unavailable is a design decision, not a prompt one.
-
Your 95% Step Accuracy Is a 60% Agent
Compounding explains part of the gap between a green eval dashboard and a broken agent. The rest is that per-step scoring counts a step as correct when it succeeds on a premise that was already wrong.