How to — put a human in the loop without making them a rubber stamp

You want a person to actually review the risky things your agent does — not to have a person's name attached to them after the fact. The way to get that is to design the checkpoint around the specific action being taken, rather than around a general sense that someone is watching.

Most teams that say "a human approves it" have built an approval theatre. The reviewer sees a fluent summary the model wrote about what it plans to do, the queue has forty items in it, and the stakes feel low because the summary reads like a competent colleague's note. Within a week the reviewer is clicking approve on the summary, not on the action. That is not a discipline problem you can fix by telling people to be more careful. It is a design problem, and it has a design fix.

The structural half is worse than the attention half. A September study of AI coding-agent harnesses found six distinct ways the action a human approves and the action the machine executes drift apart — the scope widens, the arguments change, the timing slips past the grant window, a different tool is called, the approval is delegated forward, or the meaning of the approved step is silently reinterpreted. The authors call it approval laundering, and the point is that it is reproducible rather than exotic, per the paper's authors. A reviewer can do everything right and still have authorised something that never ran.

So the job has two halves: give the human something worth judging, and make their judgement bind to what actually happens. We covered why oversight fails in practice in The human in the loop was the weakest link; this is the version you can build on Monday.

1. Tier the decisions before you build any queue.

Write three lists. The first is actions that need a human every time: money leaving the account, deleting or overwriting data, anything that reaches a customer or the public, anything that changes permissions or credentials, anything irreversible. The second is actions that need a human on a sample: drafting, summarising, internal search, low-value write operations you can roll back. The third is actions that need nobody, and it should be the longest list.

The reason to do this on paper first is that a loop over everything is a loop over nothing. If the reviewer's queue mixes a $40,000 refund with a calendar invite, they will develop one click speed for both. Oversight capacity is a fixed budget; spend it on the twenty decisions a week that can hurt you.

2. Show the artifact, not the agent's account of the artifact.

The single most common defect: the review screen contains a summary generated by the same model whose action is being reviewed. Every error in the action is faithfully described as correct in the summary, and the reviewer's judgement is only ever as good as the thing they were shown. Put the raw material in front of them — the actual diff, the actual recipient list, the actual amount and account, the actual clause that will be sent.

If the artifact is too large to read, pin the exact numbers, names and paths alongside the summary — the reviewer should be able to reject on a fact, not on a tone.

3. Bind the approval to one action, with the arguments frozen.

An approval should be a statement about one concrete action — this recipient, this amount, this file, this tool — and it should expire quickly. Session-wide grants and "allow this kind of thing" toggles are where laundering lives: the human approved a category, and the machine spent it on a member of the category nobody looked at.

Practically, that means the review screen shows the exact parameters that will be dispatched, the executor re-checks those parameters against the approval at the moment of execution rather than at the moment of clicking, and a mismatch stops the action instead of proceeding. Approvals that sit unspent for hours should lapse. Cheap to build, and it closes the drift the paper documents.

4. Make rejection cheap, and measure how often it happens.

An approval rate near 100 percent is not evidence of a good agent. It is evidence of a reviewer who has stopped reading, or a queue filtered so hard that only obviously correct items reach it. Track the override rate per action type and watch it the way you watch an error rate: near zero is a warning, and so is a sudden spike.

Then test the reviewer instead of trusting them. Seed known-bad items into the queue on a schedule — a wrong account number, a payload that would send data to an external address, a supplier name that is one letter off a real one — and record whether the reviewer catches them. You are not grading people. You are measuring whether the checkpoint still functions, which is exactly what an audit will ask you to demonstrate.

5. Budget the reviewer's time, and rotate the job.

Give the queue a length cap and a review window that matches a human's attention: a handful of items, reviewed in one sitting, at a time of day when the reviewer is not also doing their real job. Items that pile up overnight get batch-approved at nine in the morning, and a batch approval is a rubber stamp with extra steps.

Rotation matters more than it sounds. The researchers behind AI Agents Push Humans Out of the Loop argue that extended reliance on automation erodes the very skills oversight depends on — a reviewer who only ever sees the agent's output slowly loses the ability to judge it unaided. Rotating people through the role keeps that judgement alive.

6. Log the human decision separately from the machine action.

Two records, joined by an identifier: what the person approved and when, and what the system executed and when. Review the join monthly for gaps. This is what lets you answer "who signed off on this" for any action in the last quarter, and it is the only way to notice that approvals are being spent on actions nobody described to the approver.

If you are still deciding what the agent may do in the first place, that is the earlier problem — deciding what an AI agent is allowed to do sets the boundary this checkpoint sits on. AI 101 — What is an AI agent? covers the machinery if you need the vocabulary first.

Don't do this: don't count the click as the control. An approval is evidence that a person saw a screen, nothing more. If you cannot show what they saw, what they approved, and what ran, you have a signature and not an oversight mechanism — and under a serious regulatory regime that distinction is the whole requirement. Europe's AI Act expects oversight measures to let the reviewer understand the system's limits, stay alert to automation bias, and stop the system, per Article 14; NIST's AI risk framework treats how humans are configured into the system as a design decision rather than an afterthought, per the framework. Neither is satisfied by an audit trail of clicks.

How you'll know it worked: your seeded bad items get caught at a rate you can quote, your override rate sits somewhere between never and always, and for any action from last month you can produce the artifact the reviewer saw, the parameters they approved, and the parameters that executed — matching. If any of those three is missing, the human is in the loop on paper only.

What finally made your reviewers start rejecting things — the seeded failures, or a real incident? Write to us via our contact page.