Your Agent's Dry Run Doesn't Bind the Real Run — Confirm the Plan Hash, Not the Intent

A human approves a preview; the agent then executes a freshly computed call. Everything that shifts in between is unguarded — and the fix is a redeemable plan object, not a sterner confirmation prompt.

Your Agent's Dry Run Doesn't Bind the Real Run — Confirm the Plan Hash, Not the Intent

An ops agent is asked to clean up stale storage snapshots. It calls a preview tool, shows a human three snapshots older than ninety days, and waits. The human types "yes." The agent then calls the delete tool — and removes a different set. Nobody lied, nothing was injected, and every guardrail fired exactly as designed. The approval simply wasn't attached to anything.

This is the most common shape of an agent guardrail that looks solid in a demo and leaks in production: the thing a human approves and the thing the agent executes are two separate tool calls, with an unguarded gap between them.

🔍 Why the gap exists at all

A preview-then-confirm flow is really three moves:

  1. The agent calls list_stale_snapshots(days=90) and gets a set back.
  2. The set is rendered to a human, who approves.
  3. The agent calls delete_snapshots(...) with arguments it produces now.

Step 3 does not inherit anything from step 1. The model re-emits arguments from context, and the tool re-resolves them against a world that has moved on. The approval was a token of human sentiment about a rendered list; the execution is a fresh, independently computed action. The only thing connecting them is the model's intent to be consistent, which is not a security property.

The three ways the two runs diverge

Distinguishing these matters, because they need different fixes.

World drift. The preview was a query. Between the query and the delete, a nightly job created eleven more snapshots that now match days=90. If the delete tool re-runs the filter rather than acting on the set, it silently swallows the new ones. The human approved three items and authorized a category.

Argument drift. The model regenerates the call and the arguments are not identical. A snapshot id gets transposed, days=90 becomes days=9, a region qualifier present in the preview is dropped from the delete. Long contexts, summarized histories, and any step where an intermediate agent restates the plan in prose all make this more likely. Prose is a lossy carrier for arguments.

Scope drift. The agent decides to be helpful. The human approved the three snapshots; the agent notices two orphaned volumes attached to them and includes those "to finish the cleanup properly." This is the failure people expect and the least common of the three, which is why teams that only defend against it stay exposed.

The fix: hand back a plan handle

Stop treating the preview as a rendering step. Treat it as an object the system creates, stores, and later redeems.

# Step 1: the agent plans. The server owns the result.
plan = plan_delete_snapshots(days=90)
# -> { plan_id: "pl_7fa3", items: [...3 ids...],
#      digest: "sha256:9c1e...", expires_in: 900 }

# Step 2: the human approves plan_id "pl_7fa3" specifically.
#         The UI renders items from the stored plan, not from the chat.

# Step 3: the agent redeems it. Note what it cannot pass.
apply_plan(plan_id="pl_7fa3", digest="sha256:9c1e...")

Three properties do the actual work here:

  • The apply call takes no action arguments. There is no days, no id list, nothing for the model to re-render incorrectly. Argument drift becomes structurally impossible rather than merely unlikely.
  • The digest is computed by the server over the exact item set and re-verified at apply time. If the resolved targets changed, the digest no longer matches and the call fails closed.
  • The plan expires. An approval from forty minutes ago is not evidence about the world now.

Scope drift falls out for free: an agent that wants to also delete the orphaned volumes has to produce a second plan, which gets its own approval.

⚠️ What "fail closed" has to mean

The failure path is where this design usually gets undone. When apply_plan rejects a stale digest, the tempting response is to have the agent re-plan and re-apply automatically, since "the user already said yes to cleaning up snapshots." That converts a bounded approval back into a standing one and reintroduces the original bug with more machinery.

The rejection must return to the human with the diff, not the retry:

Plan pl_7fa3 no longer matches. Approved 3 snapshots; 11 now match the same filter. Re-approve the new plan or cancel.

Showing the delta is what makes this a guardrail rather than a speed bump. A human who is told only "the plan expired, approve again" will approve again reflexively.

Where to apply it, and where not to

Binding costs a round trip and a stored object. Spend it where the action is irreversible or wide.

✅ Bind the plan for deletes, bulk updates, outbound messages to real recipients, payments, permission changes, and anything that touches more rows than a human counted.

❌ Don't bind reads, idempotent writes to a single known key, or actions inside a sandbox that gets discarded. A confirmation step on a harmless action trains people to approve without reading, which raises risk everywhere else.

A useful test when you're unsure: if the tool's blast radius is determined by a filter rather than by an explicit list of targets, it needs binding. Filters are the mechanism by which "three things" quietly becomes four hundred.

The underlying principle

Approval is not a mood, and it is not a sentence in a transcript. It is a signature over a specific artifact. Any design where the artifact is re-derived after the signature is collected has an approval that means nothing precise — the model is being trusted to reproduce, from memory, the exact thing a human agreed to.

Most teams reach for a stricter confirmation prompt when this bites them: more explicit wording, a required "yes, delete 3 snapshots" echo. That treats a state-management bug as a persuasion problem. The prompt is not the weak link. The missing object is.

Keep reading

Similar posts

Matched on shared tags and category — the more bars, the stronger the overlap with what you just read.