Test AI Against Your Reviewers With Planted Errors Before Adopting It

A company can learn whether AI improves a GxP task, such as reviewing a batch record or drafting an SOP, before relying on it. QA plants known mistakes in real documents, gives them to reviewers with and without the AI, and counts what each group catches. About 200 planted errors per group are enough to tell a process that catches 80% of them from one that catches 90%.
That count answers the question an inspector will ask. The company answers for every record the AI touches, so it needs a dated test showing the new process does at least as well as the old. The European Commission's draft GMP Annex 22, consulted on in 2025, writes that bar down for the AI models it covers. A model's acceptance criteria "should be at least as high as the performance of the process it replaces", which "implies, that the performance should be known". Few companies know how many errors their own reviewers miss, so the test measures both groups.
FDA's rule for supplier results is an analogy: check before relying, then check again
No regulation says how to test AI on document work, but a familiar drug rule offers an analogy. Under 21 CFR 211.84(d)(2), a firm may use a supplier's test results in place of its own testing. It must first establish "the reliability of the supplier's analyses through appropriate validation of the supplier's test results at appropriate intervals".
FDA enforces the rule often, stating it in 233 of the 540 drug manufacturing (CGMP) warning letters on its pages dated January 2021 to 10 November 2026. Most went to firms that took a supplier's certificate of analysis on trust. In 187 of them, FDA asks how the firm will establish reliability "through initial validation" and periodic re-validation.
The rule says nothing about AI, so it serves only as an analogy. AI output, like a supplier's result, is work the firm did not do itself. The same two steps fit: check it before relying on it, then check it again at intervals.
The rule that does govern computer output sets no standard for how much checking is enough. The CGMP rule for computer systems, 21 CFR 211.68(b), says only that output "shall be checked for accuracy", as often as the system's "complexity and reliability" require. All 34 letters in the set that cite 211.68(b) concern who could change or delete data, and none cites that sentence. With no FDA benchmark, a firm's own measurement is the basis it can defend.
540 drug CGMP warning letters, Jan 2021 to 10 Nov 2026233 state the rule for relying on a supplier's results25 of 25 sampled matches read by hand were the rule128 quotes checked word for word against the letters
Open a row to see the letters behind its count. The highlight marks the words the count rests on.
24 of 187 letters, newest first. The wording repeats letter to letter.
FDAIf you intend to accept any results from your supplier’s COA instead of testing each DMG lot for strength, quality, and purity, specify how you will robustly establish the reliability of your supplier’s results through initial validation as well as periodic revalidation.
FDA's letterFDAIf you intend to accept any results from your supplier’s COA instead of testing each component lot for strength, quality, and purity, specify how you will robustly establish the reliability of your supplier’s results through initial validation as well as periodic revalidation.
FDA's letterFDAIf you intend to accept any results from your supplier’s COA instead of testing each component lot for strength, quality, and purity, specify how you will robustly establish the reliability of your supplier’s results through initial validation as well as periodic re-validation.
FDA's letter
Every card is in the data (JSON); the yearly counts are in the CSV below.
FDA and EU drafts ask firms to test the reviewer and the AI together
Regulators' AI texts ask for this comparison on the whole process, person included. FDA's January 2025 draft guidance on AI in drug development asks for confidence intervals and for test data kept apart from the data that built the model. Where a person reviews the output, the evaluation should "consider the performance of the human-AI team" (our walk-through covers the rest). The FDA and EMA principles of January 2026 likewise ask firms to "evaluate the complete system including human-AI interactions".
Draft Annex 22 adds rules for the test itself. The test set should be "sufficient in size to calculate the test metrics with adequate statistical confidence". Staff who have seen it should not help build or tune the model. FDA's final computer software assurance guidance, issued on 3 February 2026 for device makers, scales testing to risk and names AI tools in its scope.
None of these texts prescribes a planted-error test or fits document work exactly. The FDA draft leaves out internal uses such as drafting a submission. Annex 22 would bar generative AI, the kind that writes text, from critical GMP uses. A planted-error test applies their shared principle to document work and puts a number on the decision.
A planted-error test borrows a control QA already runs on visual inspectors
QA already qualifies visual inspectors this way. FDA's 2021 draft guidance on visible particulates in injectables describes test sets that include "samples created to simulate production defects", with the defects "masked to test subjects". It says inspection machines "should be validated to meet or surpass human inspection capabilities". Clinical laboratories likewise must test proficiency samples "in the same manner as patients' specimens" under 42 CFR 493.801.
A test for document work follows the same pattern. QA picks one task and defines its context of use, FDA's term for what the AI does and which decision its output feeds. (GAMP, the industry's validation guide, calls it the intended use.) QA then plants the errors that matter: a wrong limit in an SOP, an unrecorded deviation in a batch record, an eligible study left out of a literature screen. One group of reviewers works alone and another works with the AI. Nobody who configures the AI sees the answer key.
The test counts planted errors caught, errors that reach an approved document, and time per document. The AI-assisted process earns the task when it catches more and lets no more through. The plan, results and decision go into the validation file like any other qualification.
About 200 planted errors per group tell 80% from 90%
A small test cannot tell a real improvement from luck. Suppose reviewers alone catch 80% of errors and reviewers with AI catch 90%. To show that gap reliably, each group needs about 199 planted errors, 398 in all.
A test with 30 errors per group is mostly noise. If reviewers alone catch 24 and reviewers with AI catch 27, the difference is well within chance. Even when the real gap is 10 points, a test that size detects it only about one time in five. Spreading errors over many documents and reviewers keeps one person's habits from deciding the result.
The test also has to plant errors in the AI's own drafts
Planting errors only in human drafts misses the risk AI adds: a reviewer who trusts it too much. Automation bias is the tendency to accept a machine's suggestion without checking it. In a 2023 study in Radiology, 27 radiologists read mammograms with a purported AI that was deliberately wrong on 12 of 40 cases. Inexperienced readers rated 79.7% of cases correctly when the AI was right and 19.8% when it was wrong. Very experienced readers fell from 82.3% to 45.5%. A review of 40 studies found the bias in single tasks whose output took effort to check, as a long document does.
So QA should plant some errors in AI drafts after the AI writes them. It then compares how often reviewers approve those with how often they approve errors in human drafts. If reviewers pass AI errors more often, the process is not ready for that task, whatever its overall catch rate. Our post on human-only review covers the baseline.
A result holds only for the model that was tested
The test measures one model, so a change to the model reopens the question. Draft Annex 22 asks that any change to a model be evaluated "to determine if the model needs to be retested". The FDA and EMA principles ask for "periodic re-evaluation". QA should keep the planted-error set under its own control and run it again after every model change. Change control for AI models and qualifying AI vendors cover the notice a supplier agreement should require.
Questions and decisions for the steering committee
- Which GxP task do we test first, and what is its context of use?
- Who in QA builds about 200 planted errors per group, some in AI drafts?
- What bar must AI-assisted work clear on errors caught, errors approved and time?
- Do vendor agreements require notice of model changes, and who re-runs the test?
Frequently asked questions
Do we need to validate ChatGPT or Copilot before staff use it on GxP work?
You validate each GxP use of it. Keep a test on your own documents showing the AI-assisted process does at least as well as the one it replaces.
Who is accountable if the AI misses an error?
The company and the person who signs the record, as with any other system. The test is the evidence that the process they signed under was fit for the task.
What do we tell the auditor?
Show the test plan, the planted-error results for both groups, the adoption decision, and the date of the last re-run after a model change.
Data and method
We read the warning letters on FDA's pages dated January 2021 to 10 November 2026 and kept the 540 drug CGMP letters. These cite a Part 211 section other than consultant boilerplate and come from offices other than food, devices, tobacco and veterinary medicine. A letter counts when it states the requirement to establish a supplier's reliability before accepting its results. All 25 matches we read at random were that requirement, and we read all 34 letters citing 211.68(b). FDA removes some letters from its pages, so the counts cover letters still published. The sample sizes use the standard comparison of two proportions at 5% significance and 80% power, checked by exact calculation.
Downloads: letters by year (CSV), planted-error sample sizes (CSV).