How to audit what an AI agent did in your books

A reviewer asks a simple question about an entry posted last month: why was this coded to that account? If the answer is "the system decided", the control has failed, however accurate the system is.

This article describes how we think about the record an agent should leave, and how a reviewer can use it. It covers the practice, not audit methodology, which belongs to your auditors.

The problem we set out to solve

People leave traces without trying: an email, a note, a sign-off. Software leaves a trace only if somebody designed it to. An agent that reads a document, weighs context and posts an entry makes several decisions in a few seconds, and without design none of them is recorded.

The aim is that any action can be explained later by someone who was not there.

Design principles

• Auditability by default. Logging is not an option to enable. An action with no record does not happen.
• Accuracy over speed. When unsure, hand off to a person.
• Human review where judgment lives. Limits are written down and enforced in the system, not in a policy document alone.

What the record should contain

For each action, store these items together, so one lookup returns the whole story:

1. Inputs. The source documents or data, preserved as they were read.
2. The decision. The entry proposed or posted, with the account, amount and period.
3. The basis. The rule applied or the reasoning the model gave, in plain text, plus a confidence value if the system produces one.
4. Authority. Which limit allowed the action, and whether a person approved it.
5. Identity. A dedicated service identity for the agent, kept separate from any person's.
6. Version. Which version of the model, prompt and rules was in force.
7. Time. A timestamp for each step.

The version matters more than teams expect. When behavior changes after an update, the version tells you the date it began.

Keep the log append-only, so nobody can edit past entries. Retain it for as long as your records policy requires for the underlying transactions.

How a reviewer uses it

Sample with intent

Random sampling is a start. Stratify it too. Take items from each category the agent handles, from the low-confidence band, from the largest amounts, and from the days right after any change to rules or model version.

Review a fixed set of questions

For each sampled item, ask: were the inputs complete, was the decision consistent with policy, was the authority sufficient, and would a careful person have done the same? Record the answer to each, so the review itself leaves a trace.

Track the error type

A wrong account is different from a wrong amount, which is different from a missed duplicate. Count by type. The mix shows what to fix and which limit to tighten.

Feed findings back

A finding should end in a change: a narrower limit, a new rule, a category moved to human approval. Log the change as you would any control change.

Test the agent as you would a new hire

Seed a handful of known-bad items, such as a duplicate or an invoice dated into a closed period, and see if the system catches them. Do this after each model or rule update. It is the cheapest regression test for a control.

For a broader risk vocabulary, the NIST AI Risk Management Framework and the OWASP Top 10 for LLM applications help you ask about what could go wrong. The second one is relevant when an agent reads documents that outsiders control, since a crafted document can try to steer it.

What this means for your team

You do not need a new department. You need a monthly routine with an owner, a sample method and a place to record findings. A half day per month is a fair start for a single workflow.

Begin by asking your current or prospective system for the record of one transaction. If it cannot produce the seven items above, you know what to request next.