#human-in-the-loop
Every post tagged "human-in-the-loop".
16 posts · showing 1–10
-
Content Boundaries: What Your Agent Should Refuse, and How to Test It
Providers handle the general case. Scope, regulatory limits, commitments, and how your agent represents you are yours — and the test suite needs both halves, or you'll ship something that refuses everything.
-
Eval Cases Where the Right Answer Is "I Don't Know"
Suites measure whether the agent got it right, never whether it should have answered at all. Five case types where the correct response is a question — plus the paired cases that stop you from just making it timid.
-
Getting an Agent to Say How Sure It Is
Ask a model how confident it is and you get a plausible number that doesn't move with difficulty. Five signals that actually correlate with being right — and the calibration check that tells you if yours is noise.
-
The Documentation Agent That Actually Kept Docs in Sync
Ask an agent "do the docs need updating?" on every PR and you get noise reviewers close unread. Trigger on contract changes, forbid new prose, cite the source line — and run the corpus-wide contradiction check first.
-
Delegating to a Human as If They Were an Agent
Same interface, same contract, same result shape — a person is a worker with hours of latency and much better context. Which makes suspendable runs the prerequisite, not a nice-to-have.
-
State Machines for Agents in Regulated Flows
When doing steps out of order is a compliance failure regardless of outcome, a model-decided control flow is the wrong shape. Put the ordering in code, the conversation in the agent, and scope the tools per state.
-
The Migration Agent: 4,000 Files and What Made It Tractable
The difference between a migration agent that saves weeks and one that produces four thousand unreviewable diffs is decided before it touches a file — and it starts with building the checker, not the agent.
-
Where Agents Don't Belong (Yet)
Six task shapes where the agent architecture is the wrong fit regardless of model quality — each with a tell you can check in an afternoon, before committing a quarter to it.
-
When the Worker Needs to Ask the Supervisor a Question
A blocked worker with no channel back will guess, and the guess is invisible in what it returns. Making "needs input" a result type costs nothing — and the questions it logs are a to-do list for your delegation template.
-
The Data-Cleaning Agent and the Column It Silently Dropped
It normalized the phone numbers, standardized the country codes, and quietly deleted 412 rows that looked untidy. Any task phrased as "make this better" reads as permission to remove what doesn't fit.