How FDA's Draft AI Guidance Judges Whether a Model Can Be Trusted

FDA's draft guidance on AI in drug development asks a sponsor to rate an AI model's risk, then prove the model credible in proportion to it. The rating rests on how much the decision depends on the model and on how bad a wrong decision would be, and the sponsor controls only the first. Designing the model's role is therefore the first and cheapest decision a sponsor makes, because it sets how much evidence FDA will expect.
The guidance is Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products, published in January 2025 under docket FDA-2024-D-4689. In November 2026 FDA's page still lists it as a draft, yet FDA has said it will apply the draft in an AI pilot for early-phase trials. Its two worked examples stop at step 3 of a seven-step process, so this post follows one of them through to the end.
The draft covers models whose output FDA relies on
The draft applies when an AI model's output supports a regulatory decision about a drug's safety, effectiveness or quality. That covers nonclinical work, clinical trials, postmarketing safety and manufacturing. Two uses are left out. One is drug discovery. The other is "operational efficiencies (e.g., internal workflows, resource allocation, drafting/writing a regulatory submission)" that do not affect patient safety, drug quality or the reliability of study results. A tool that drafts a submission is therefore outside the framework, although the firm still answers for what it submits, as FDA's first AI warning letter showed.
For a model inside the scope, the draft asks for seven steps:
| Step | What the sponsor does | What it produces |
|---|---|---|
| 1 | Define the question of interest | The decision the model informs |
| 2 | Define the context of use (COU) | The model's role and scope, and what other evidence answers the same question |
| 3 | Assess model risk | A rating from model influence and decision consequence |
| 4 | Plan the credibility assessment | A credibility assessment plan, sized to the risk |
| 5 | Execute the plan | Test results |
| 6 | Document results and deviations | A credibility assessment report |
| 7 | Decide whether the model is adequate for the COU | Accept, add evidence or controls, change the model, or revise the COU |
FDA wants to discuss steps 3 and 4 before the work begins. The plan a sponsor brings to an early meeting should at least state the question, the context of use, the risk rating and the proposed credibility activities.
Risk depends on the model's influence and the cost of a wrong decision
Model risk, in the draft, is the chance that the model's output leads to a wrong decision with an adverse outcome. Model influence is how much the model's evidence counts against the other evidence on the question. Decision consequence is how bad a wrong answer to the question would be. The draft rates each factor separately, and the overall risk rises as either one rises.
Only influence is in the sponsor's hands. Decision consequence comes from the question itself, so a sponsor cannot lower it without asking a different question. Model influence comes from the context of use, which the sponsor writes. FDA's two examples show the difference:
| Drug A: clinical trial | Drug B: manufacturing | |
|---|---|---|
| Question of interest | Which participants are at low risk of a life-threatening reaction and can skip 24 hours of inpatient monitoring after dosing? | Do vials of an injectable meet the fill volume specification? |
| The model's role | Sorts participants into low- and high-risk groups from baseline characteristics and lab values | Checks the fill level of every vial from images |
| Other evidence on the question | None: the model alone decides who goes home | Release testing measures fill volume on a sample from every batch |
| Model influence | High | Low |
| Decision consequence | High: a misclassified participant could react at home without treatment | High: wrong fill volume can cause dosing errors |
| Model risk | High | Medium |
Both questions carry a high consequence, and the ratings differ only because of influence. The vial model shares the decision with an independent test, so release testing can catch a wrong reading before the batch ships. The Drug A model decides alone, so nothing catches its errors before a participant goes home.
A plan for Drug A has to show the error that matters
The draft lists what a credibility plan contains but gives no example of one, because the right activities "may vary depending on the nuances of a specific development program". It is clear that a high-risk model faces the most scrutiny. For such a model FDA "may request all of the information" in the plan's categories, and its acceptance criteria should be stricter and more detailed than a low-risk model's.
For the Drug A model as FDA describes it, each part of the plan the draft lists takes a specific form:
- The model. Its inputs (baseline characteristics and lab values), its output (a low- or high-risk label), its architecture and parameters, and why that approach was chosen.
- The development data. Which trials the training and tuning data came from, how participants were labeled as having had the reaction, and whether those participants resemble the new trial's population. Earlier trials monitored every participant as an inpatient for 24 hours, so they should have recorded the reactions in that window, which makes them a natural source.
- The test. Test data independent of the development data, which the draft suggests could come from a different trial, and performance metrics with confidence intervals. For this question, the number that matters is how often a participant who later reacts is labeled low risk, because that is the error that sends someone home unprotected. Sensitivity and negative predictive value measure it.
- Uncertainty and limits. How confident each prediction is, and where the model is likely to fail, such as in subgroups the training trials enrolled few of.
The sponsor then runs the plan (step 5) and writes the results, including any departures from the plan, into a credibility assessment report (step 6). The draft leaves it to discussion with FDA whether that report goes into a submission or meeting package or is kept on file for inspection.
If the model falls short, the draft lets the sponsor change its role
At step 7, the sponsor or FDA decides whether the model is credible enough for its risk. When it is not, the draft offers these ways forward:
- add other evidence, so the model has less influence;
- test more rigorously or add development data;
- put controls around the model;
- change the modeling approach;
- reject or revise the context of use.
The first option repeats the lesson of the vial example. A Drug A design could keep the model but limit its role. An investigator could confirm each low-risk label, or participants with uncertain predictions could stay as inpatients. Either way the model has less influence over who goes home. The draft does not say what rating such a design would get, so a sponsor should propose it, with its reasoning, at an early meeting such as INTERACT or a pre-IND meeting.
Models that keep running need a monitoring plan
FDA typically assesses AI evidence on locked data, produced by a model at one point in time. A model that keeps running for years, as in manufacturing, is different, because its performance can drift as its inputs change. For those models the draft asks for ongoing performance monitoring, as often as the model's risk warrants, and for every change to go through the manufacturer's change management system. A change that affects the model's performance is reported to FDA under the usual postapproval change rules. A sponsor can also agree in advance which model changes need no prior submission, using the established conditions of ICH Q12.
The draft does not say how often to monitor or what level of drift should trigger retesting. Those were the questions commenters raised most, as our analysis of the 124 public comments shows, and FDA has not yet answered them.
FDA's later documents keep the same framework
FDA has published two documents on this subject since the draft, and both build on it. In January 2026, FDA and EMA issued ten guiding principles of good AI practice in drug development. The second principle sums up the draft's framework, asking for "proportionate validation, risk mitigation, and oversight based on the context of use and determined model risk". In April 2026, FDA asked for comments on a pilot of AI in early-phase trials and said it would apply the considerations in the draft. EMA's own reflection paper on AI, adopted in September 2024, remains its main text.
A sponsor planning an AI model for a trial or a plant therefore has a working standard now, even though the guidance is not final. Write the question and the context of use first, and rate the risk with the reasoning shown. Then take both to FDA before building the credibility plan, because the context of use decides how large that plan has to be.
Frequently asked questions
Is FDA's AI guidance for drugs final?
No. In November 2026 FDA's page still marks the January 2025 guidance as a draft, but FDA's April 2026 pilot notice says it will apply the draft's considerations.
Does the guidance cover AI that drafts regulatory documents?
No. It excludes AI used for operational efficiencies and names drafting a regulatory submission as one. The exclusion holds only while the use does not affect patient safety, drug quality or the reliability of study results.
How do I lower my AI model's risk rating?
Reduce the model's influence by adding independent evidence on the same question, such as a confirmatory test or a human review. That is why FDA rates its manufacturing example medium, although a wrong answer there would also be serious.