Review that is added, and review that is designed
Keeping a human in the loop is the standard reassurance offered about AI in business processes. Often it means an approve button was added at the end. The person behind that button sees a recommendation, has forty more waiting, and learns within a day that approving is quicker than checking. The control exists on paper only.
Designed review is different. It starts from the question of where a person's judgement adds something the machine lacks, and it gives that person the time, the information and the means to exercise it. Fewer items reach a reviewer, and each one is really looked at.
Sort each step by what an error would cost
| Level | An error would be | Examples | Review design |
|---|---|---|---|
| Low | Cheap, visible and easily undone | Tagging an email, suggesting a category, drafting internal notes | No approval. Sample afterwards |
| Moderate | Annoying or embarrassing, but correctable | A reply to a routine enquiry, a calendar booking, a data entry draft | Approval at first, then only for cases the system flags as uncertain |
| High | Costly, or affecting a customer's rights or money | A refund, a credit decision input, a contract clause, a price | Approval by a qualified person, every time |
| Critical | Irreversible, unsafe or regulated | Payments above a limit, medical or legal conclusions, deleting records | AI assists only. A person decides and acts |
Classify every step of the workflow with the business owner and, where relevant, compliance. Some decisions about individuals are subject to legal rules on automated decision-making in certain jurisdictions, so have those steps reviewed by a legal adviser. The classification, written down, becomes the specification for where review happens.
Give the reviewer what they need to judge
A reviewer shown only a conclusion can only agree with it. To judge, they need to see what the AI saw and why it proposes what it does, without leaving the screen.
- The proposal, clearly marked. What will happen if they approve, in plain words: send this message, pay this amount, update these fields.
- The evidence beside it. The source document, the customer record, the passages relied on, with the relevant parts highlighted.
- What the system was unsure about. Missing data, conflicting sources, low confidence in an extracted value. Draw attention to those, not to the easy parts.
- What changed. For edits to a record or a document, show old and new side by side.
- Editing in place. The reviewer should be able to correct a figure or a sentence and then approve, not only accept or reject the whole.
- A reason on rejection. A short choice of reasons plus free text, captured for improving the system.
Keep review from becoming a formality
Review that stays real
- Only items that need judgement reach a person
- Volume per reviewer leaves time to look
- Reviewers are qualified for the decision
- Known test cases with errors are mixed in occasionally
- Corrections are easy and visibly used to improve the system
Review that decays
- Everything goes to review, so nothing is examined
- Approval is one click and correction is ten
- Reviewers are measured on speed alone
- The same person builds, reviews and reports
- Approve all exists as a button
The clearest measure is the override rate: how often reviewers change or reject what was proposed. A rate of zero over a long period means either the AI is perfect on those cases, which should be confirmed by an independent sample, or people have stopped looking.
Design the paths around the happy case
- 01
What does the system do when it is unsure?
What to decideIt should route the item to a person with the reason, not guess. Define the signals that trigger this.
- 02
Who reviews what?
What to decideMatch decision types to roles and authority limits. A refund above a set amount goes to a supervisor.
- 03
What if nobody reviews in time?
What to decideA waiting item needs a deadline, a reminder and a fallback. The safe default is usually to do nothing and notify.
- 04
How does a reviewer pass something up?
What to decideA second-level route for cases beyond their authority or knowledge.
- 05
How does a customer reach a person?
What to decideWhere the workflow faces customers, they need a visible way to ask for human handling.
- 06
How is the whole thing paused?
What to decideA named person must be able to stop automatic processing at once, with work falling back to the manual route.
A worked example
Review design for supplier invoice processing
This example is invented to show review placed by risk. A manufacturer uses AI to read supplier invoices and prepare them for payment.
- Steps and levelsReading the invoice fields: low. Matching to a purchase order: moderate. Approving for payment: high. Changing a supplier's bank details: critical.
- AutomaticInvoices that match a purchase order and goods receipt exactly, under a set amount, pass straight to the payment run. A sample of 1 in 20 is checked afterwards.
- To reviewInvoices with a mismatch, a missing order number, a low-confidence amount or a value over the limit go to an accounts clerk, who sees the invoice image beside the order, with differences highlighted.
- Never automaticAny change of bank details is flagged and handled by a senior clerk who confirms by phone with the supplier. The AI only points out that the details differ.
- MeasuresIn the first quarter, clerks correct about 1 in 8 of the invoices they review, mostly tax amounts on foreign invoices. That pattern is fed back and the rate falls.
Clerks handle a third of the invoices instead of all of them, and those are the ones that need a person. The override rate shows the review is real, and the bank details rule blocks a common type of fraud whatever the AI does.
Record every decision and measure the review
For each item, keep what the AI received, what it proposed, who reviewed it, what they changed and when. That record answers the auditor's question, supports any later dispute and is the raw material for improving the system.
- 01
Override rate by step and reason
Shows where the AI is weak and whether review is active.
- 02
Time in the review queue
Shows whether review has become the bottleneck.
- 03
Errors found after approval
From sampling and from complaints. This is the true quality of the whole workflow.
- 04
Share handled without review
Should rise only as evidence supports it, step by step.
- 05
Reviewer agreement
Have two people review a sample independently. Large disagreement means the criteria are unclear.
Loosen review deliberately, never by drift. When a moderate step has shown a low error rate over a meaningful period, move it to sampling, announce the change and keep the sample.
Human review design checklist
Human review design checklist
0 of 11 checkedWhat to do next
Write the steps of one workflow down the side of a page and, for each, what the worst realistic error would cost and whether it could be undone. That table tells you where a person belongs. Our AI use case assessment asks related questions about the workflow as a whole.
Our business process automation page describes how we build approval steps into automated work. Describe your workflow to us and we will suggest where review should sit.
Common questions
Does every AI output need human approval?
No. Reviewing everything is expensive and, in practice, leads to nothing being examined properly. Require approval where an error would be costly, hard to reverse or subject to regulation. For low-risk steps, let the system act and check a sample afterwards. The aim is for people to spend their attention where judgement is needed.
What is the override rate, and what should it be?
It is the share of reviewed items that the reviewer changes or rejects. There is no correct figure, but it should not be zero for long. A healthy rate reflects the difficulty of the cases sent to review. Track it by step and reason: a falling rate with stable quality suggests the AI is improving, and an unexplained drop suggests inattention.
How do we stop reviewers from approving everything?
Send them fewer, harder items. Show the evidence beside the proposal. Make editing as quick as approving. Avoid measuring reviewers on speed alone. Occasionally include known test cases containing an error and see whether they are caught. And make sure reviewers see that their corrections change how the system behaves.
Can human review be removed once the AI proves reliable?
It can be reduced for specific steps, on evidence, and replaced by sampling. Keep it for decisions that are high cost, irreversible or regulated, whatever the measured accuracy. Any reduction should be a recorded decision with a named owner, reversible if sampled quality declines, and never something that happens because the queue became inconvenient.
Who is accountable when a reviewed AI decision is wrong?
The organization remains accountable for its decisions, and internally that should rest with the owner of the process, not with the reviewer alone or the software. Clear roles, authority limits and a complete record of what was proposed and approved make it possible to learn from an error without searching for someone to blame.
