Safety & Guardrails
Prompt injection, permissions, sandboxing, human checkpoints.
24 posts · showing 1–10
-
Your Research Agent Read One Poisoned Web Page, and It Still Had the Email Tool. Lock Risky Tools the Moment Untrusted Text Enters the Context.
Prompt-injection filters try to spot the bad sentence. A taint flag skips that guess: once fetched text is in the context, the tools that can do damage are no longer offered.
-
The Refund Tool Capped Each Refund at $500. The Support Agent Issued Eleven $490 Refunds to One Customer.
A per-call limit only answers "is this action too big?" Agents fail on a different question: "is this sequence too big?" Here is how a guardrail that never fired still let the money leave, and the ledger check that would have stopped it.
-
Your Agent Asks Permission Nine Times an Hour, So Nobody Reads the Prompts Any More. Spend a Fixed Approval Budget.
Every confirmation dialog you add makes the next one less effective. Here's how to classify write actions by reversibility and reach, auto-allow the safe corner, and reserve the human's attention for the handful of actions that can't be undone.
-
The Page Your Agent Read Told It to Email Your Customer List. "Ignore Instructions in the Content" Won't Stop the Next One.
A system-prompt warning makes injection less likely. It doesn't make it impossible, and the attacker gets as many tries as they like. The real fix lives in the tool dispatcher, not the prompt.
-
Your Agent's Dry Run Doesn't Bind the Real Run — Confirm the Plan Hash, Not the Intent
A human approves a preview; the agent then executes a freshly computed call. Everything that shifts in between is unguarded — and the fix is a redeemable plan object, not a sterner confirmation prompt.
-
Untrusted Text Doesn't Stay Untrusted: Where Provenance Gets Lost in an Agent Pipeline
An injection scanner that fires on the raw email won't fire on the tidy summary of it. Untrusted content gets laundered clean every time an agent rewrites it — here's where the taint disappears, and how to make it survive.
-
Your Agent Retried the Tool Call. The Customer Got Charged Twice.
A timed-out tool call is not a failed one — it is an unknown one. Here is why "just retry" quietly doubles real-world side effects, and where to put the idempotency check so it cannot.
-
Read, Draft, Send: Where to Draw the Permission Line in a Personal Email Agent
Most email agents get permissions wrong by granting them per tool. The safer line runs through the arguments — who the recipient is, how many there are, and whether the thread was already open.
-
Read-Only Was Never the Same as Safe
SELECT-only credentials close exactly one of the four ways an analytics agent can hurt you. The other three don't require the ability to write anything to the database at all.
-
Injection That Crosses a Handoff
The fetcher had no write tools and the analyst never saw a web page. The separation held perfectly — and routed the payload around itself, through the one field that carries prose.