When an AI Vendor Swaps Its Model, Treat It as a GMP Change

When an AI vendor moves its tool to a new model, the tool's answers change. Under GMP rules that is a change to a computerised system, and the company using the tool has to assess it. Providers do not always leave time for the assessment. Since September 2024, Google's changelog shows 12 Gemini model names that began serving a different model, and eight of those switches took effect the day Google announced them.
Most AI tools that write or answer in words run on a large language model (LLM) from a provider such as Google, OpenAI or Anthropic. Each version of that model is like a software release, so a vendor that moves to a new version has installed an upgrade its customers may never hear about. So has a provider that re-points the model name the vendor's code calls. On October 8, 2026, Google announced that "API requests to gemini-3.7-flash are automatically routed to gemini-3.8-flash", so a tool calling the old name got the newer model's answers that day.
Providers change the model behind a name, sometimes the same day
Notice varies widely by provider. We collected every text-model retirement since 2024 from the deprecation pages of OpenAI and Anthropic and Google's Gemini changelog. OpenAI gave a median of 182 days' notice across 53 models, and Anthropic 63 days across 20. Google gave 34 days across the 18 Gemini models it shut down.
A re-pointed name is harder to catch than a shutdown, because a shutdown returns an error and a re-pointed name does not. Four of Google's 12 were aliases ending in "-latest", which Google designs to move to its newest model, so a vendor that calls one has chosen to follow it. The rest looked fixed. On May 6, 2025, Google announced that gemini-2.5-pro-preview-03-25, a name with a date in it, "will automatically point to the new version of the model". Microsoft's Azure OpenAI re-points by policy: at retirement, "the service automatically upgrades Global Standard and Data Zone Standard deployments to the replacement model". Google is the only provider whose pages record each switch, which is why its switches can be counted at all. At any provider, a model name in a vendor's code is no proof that the model behind it stayed the same.
103 text models retired or re-pointed by Anthropic, OpenAI and Google, June 2024 to November 202643 of 540 drug CGMP warning letters since 2021 that cite 211.68, all read by hand58 of 58 FDA quotes found word for word in the letter
Open a row to see the provider's own notice or FDA's sentence behind each count. The highlight marks the words the row rests on.
How providers retire text models (notice days)
12 models, newest first; median notice 0 days
Google's changelogEffective December 1, 2026, all API requests to gemini-2.5-pro will automatically route to gemini-3.8-flash and be billed at Gemini 3.8 Flash rates.
Provider's pageGoogle's changelogEffective December 1, 2026, all API requests to gemini-2.5-flash will automatically route to gemini-3.5-flash-lite and be billed at Gemini 3.5 Flash-Lite rates.
Provider's pageGoogle's changelogIt is no longer available to select in Google AI Studio, and API requests to gemini-3.7-flash are automatically routed to gemini-3.8-flash.
Provider's page
What FDA cites under 211.68 (26 letters)
Every card is in the data (JSON), and every model and letter is in the CSVs below.
GMP rules already treat a new model as a change
EU and US drug rules never mention AI models, but both control every change to a computerised system, and a new model is one. The EU's rules sit in Annex 11, in force since June 2011. Its section 10 requires that any change to a system, "including system configurations", be made "in a controlled manner in accordance with a defined procedure". Section 3 requires formal agreements with any supplier that maintains or modifies a system.
US rules reach the same place. Under 21 CFR 211.68(a), computers used in manufacturing must be checked under "a written program designed to assure proper performance". Under 211.100(a), the quality control unit approves production procedures and any changes to them. When a procedure relies on an AI tool, a new model changes what that step does, so in our reading the change belongs before the quality unit. ICH Q10 adds that "all changes should be evaluated by a company's change management system".
FDA holds the firm responsible for changes to its systems even when software made them. Of 540 drug CGMP warning letters since 2021, 43 cite 211.68, and 26 of those describe a failure of a computer system. In 6 of the 26, staff or a software update could change how the system itself worked. Ava Inc. told FDA in 2026 that "a software update allowed users to bypass" its access restrictions, and that it had turned off automatic updates. FDA still found the response inadequate. No letter among them concerns an AI tool, but a model swap is the same kind of event, and a firm should expect to answer for it.
Draft EU rules would give users a right to test a new version first
The revisions now in draft go further. Draft Annex 11 wants service contracts to agree on "the regulated user's possibility to test" new versions "prior to release". The Commission consulted on it in 2025, alongside a draft Annex 22 on AI. Neither had been adopted by November 2026.
FDA's draft guidance on AI in drug development likewise sends changes to a manufacturing model through the change management system (our summary). For devices, FDA agrees model changes in advance through predetermined change control plans. A GxP user can fairly ask its AI vendor for the same advance notice.
The supplier agreement should pin the model and require notice
Most of the control sits in the supplier agreement, because only the vendor knows which model answers each request. It should name the exact model version behind each feature and have the vendor record it with every output. It should also oblige the vendor to pass on any notice of change as soon as its provider gives it, to say whether the provider can route the name to another model, and to let you test before any switch. A vendor that will not name its model leaves no way to tell when it changed, which belongs in the supplier assessment.
Retest each new model on a fixed set of your own tasks
A fixed test set, a list of real tasks from your own work with answers your experts have signed off, shows whether a new model is fit for use. How much of the set to rerun depends on the tool's context of use, FDA's term for what its output is used for, which validation guides call the intended use. Under the risk levels we use for AI in GMP work, a tool whose output never enters a record needs a spot check. A tool that drafts record content needs the full set rerun against criteria fixed in advance, and a tool that informs a decision also needs the quality unit to approve the results before staff use it.
Retesting does not replace substantive review of each output. That review catches single errors, but only the test set shows whether the new model makes them more often. Annex 11 also asks for periodic evaluation of each computerised system, including its "upgrade history". For an AI tool, that review should check the model version on recent outputs and rerun a sample of the test set, which is how a firm finds a swap nobody reported.
A steering committee should know which tools a model swap would reach
- Which GxP processes use an AI tool, and which model version does each run on?
- Does each supplier agreement require notice and a test window before the model changes?
- Can the provider route the model name our vendor uses to a different model?
- Who approves the change record when the model changes, and which test set is rerun?
- Does each AI tool's periodic review check the model version on recent outputs?
Data and method
Retirements of text models come from OpenAI's and Anthropic's deprecation pages and Google's Gemini changelog, as saved on 18 November 2026. Notice runs from the first announcement to the day the model stopped serving or its name moved. Microsoft publishes no notice date per model, so its pages appear only as policy. The letters are the 540 drug CGMP warning letters FDA published from January 2021 to November 2026. We read all 43 that cite 211.68 and checked every quoted sentence against the letter text. The 26 letters that describe a failure also cite staff able to delete or alter records (23), audit trails off or unreviewed (15), lost data (8) and unvalidated software (6).
Downloads: model retirements (CSV), letters citing 211.68 (CSV) and letters by year (CSV).
Frequently asked questions
Do we have to revalidate every time the AI vendor changes its model?
You have to assess every change, as with any software upgrade. How much you retest depends on what the tool's output is used for, from a spot check to a full rerun of your test set.
What do we tell an inspector about the model behind our AI tool?
Name the model version the tool runs on, the change record for each version since you approved it, and the test results behind each approval.
Is it our problem if the vendor changed the model without telling us?
Yes. FDA holds the manufacturer responsible for its computer systems, so the supplier agreement should require notice and your periodic review should check for changes nobody reported.