Compliance and Audit for Agent-Written Systems

Regulated environments require evidence about how software was produced and how decisions get made. Those requirements were written assuming human authorship and human decision-making, and agent involvement doesn't fit the forms cleanly. Most of it is answerable — with specific practices that are cheap if adopted early and expensive to reconstruct.

What auditors actually ask

Underneath the paperwork, the questions are consistent:

• Who is responsible for this?
• How do you know it's correct?
• How would you know if it stopped being correct?
• What prevents the bad outcome?
• Can you show me what happened in a specific case?

None of those become unanswerable because an agent was involved. They become answerable differently, and the difference has to be designed rather than explained after the fact.

Answering each

Who is responsible. Not the agent. The named person who authorized this class of action, with this scope, under these conditions — and that authorization recorded with a date and a scope. ⚠️ An audit trail whose actor field says agent-service for every action cannot answer this question at all, which is the most common finding.

How do you know it's correct. The eval set, the oracle, the pass rates, and the safety invariants run as gates. This is a stronger answer than most human-authored systems can give, because it's measured rather than asserted — and it's worth framing that way rather than defensively.

How would you know if it stopped. Monitoring, drift detection, the sampled review, the alert thresholds. Also the re-evaluation process for model changes, since "the model changed under us" is a question auditors will learn to ask.

What prevents the bad outcome. The permission model, the approval gates, the blast radius bounds. Enforced in code rather than in prompts — a control that lives in a system prompt is not a control, and that distinction is one an auditor will understand immediately once it's explained.

Show me a specific case. The run trace, the audit record, the authorization it acted under, the version of everything in play at the time.

✅ What to have in place

Version everything, permanently. Model, prompt (by rendered hash), tool definitions, policy. For any past action you must be able to state exactly what configuration produced it. Retention for as long as the regulation requires — which is usually far longer than trace-retention defaults.

Audit records naming humans. Actor, affected entity, authorization, scope, and a reference to the run and turn.

Change control on prompts and tools. They're behavior. If code changes require review and records, so do these — and an auditor who understands that a prompt determines behavior will ask.

Documented human checkpoints, with evidence they're real. ⚠️ Approval time data matters here: a gate with a median decision time of two seconds is a documented control that demonstrably isn't functioning, and that's worse than not claiming it.

A defined process for model changes, treated as a change to a validated system rather than a dependency bump.

💡 The framing that works

Rather than presenting agent involvement as an exception to normal controls, present it as a system with the same control questions and different mechanisms.

Practically: a well-instrumented agent system can answer "how do you know it's correct" with a measured pass rate against a maintained specification, and "show me what happened" with a complete trace. Many human-authored systems answer the first with "we tested it" and the second with a partial log.

→ The instrumentation that makes agents operable is largely the same instrumentation that makes them auditable. That's a genuine alignment of interests and worth using — the eval set and the audit trail are both things you need anyway.

🔍 The rehearsal

Pick one consequential action from three months ago and produce, from records alone: what happened, who authorized it, under what policy, with which model and prompt version, what verification was in place, and what the sampled review said about quality that month.

The gaps that exercise finds are the ones an audit will find, and it takes an afternoon rather than a quarter.

The takeaway

The audit questions don't change; the mechanisms do. Name a responsible human and record the authorization, version everything permanently, put change control on prompts and tools, enforce controls in code rather than in prompts, and keep the evidence that human checkpoints are functioning. Then rehearse reconstructing one specific action from records — because that exercise is the audit, run early enough to fix what it finds.