AI in GMP: Designing AI that holds up under Annex 22

1. Where Annex 22 stands, and how much time you have

“Annex 22: Artificial Intelligence” is a proposed new annex to EudraLex Volume 4, drafted by the EMA GMP/GDP Inspectors Working Group together with PIC/S. It runs to six pages and ten sections, from Scope to Operation.

• 7 July 2025: draft published for consultation, alongside draft revisions of Annex 11 and Chapter 4.
• 7 October 2025: consultation closed.
• 30 June – 1 July 2026: EMA multistakeholder workshop. EMA noted that the consultation “suggested support for potentially enabling the use of technologies such as generative AI (GenAI) or large language models (LLMs) in medicines manufacturing”, and that it is “still considering the implications”.
• Latest status as of October 2026: the EMA is planning to submit the final text to the commission in Q4 2026, with potential adoption in H1 2027.

How much time that leaves: Annex 11 was published in January 2011 and applied from 30 June 2011; Annex 15 in March 2015, applying from 1 October 2015; Annex 1 in August 2022, applying from August 2023. Six to twelve months is enough to update SOPs and validation plans. It is not enough to replace a system.

Two implications already:

The annex’s influence will not stop at the EU border. PIC/S co-drafted the annex, and PIC/S has 57 participating authorities, including the FDA, the MHRA, Swissmedic, Health Canada, the TGA and Japan’s PMDA. FDA and MHRA were cited as observers. Expect the reasoning to travel well beyond the EU.

The GenAI exclusion is the most debated clause, and it might change. EMA’s own summary of the consultation points of July 2026 mentioned the potential enablement of GenAI and LLMs under conditions rather than excluding them outright. If it softens, it will probably move towards the FDA position: GenAI allowed where output quality can be demonstrated and controlled. In practice that means constraining a probabilistic model until it behaves deterministically or reviewing most outputs. Bottom line: if you can achieve the desired outcomes with deterministic models, it’s an easier path.

2. Where Annex 22 sits among the rules you already work under

Annex 22 is not self-sufficient: it’s a set of refined requirements for AI within the broader Computer System Validation and AI regulation regulation. Here is an overview of the key pieces of regulation.

• Annex 11 deals with the software system, Annex 22 deals with the model. Annex 11 focuses on how to validate and control software systems. It is also being reworked, with a public draft and an expected finalisation in 2027. Annex 22 asks whether the model performs well enough for this use, on this site, for this process, with thoroughly justified choices on training data and acceptance criteria.
• The EU AI Act is a separate track. It is horizontal product law with obligations by risk class. An AI Act conformity assessment is not evidence for a GMP inspector, and an Annex 22 package is no defence under the AI Act.
• Accountability stays with the regulated entity, not the vendor, not the models. An AI output is a GMP record under Eudralex Chapter 4 like any other. Documentation “should be available and reviewed by the regulated user irrespective of whether a model is trained, validated and tested in-house or whether it is provided by a supplier or service provider” (Annex 22 §2.2). 
  
  

How the EU draft compares with the US

The FDA position is less prescriptive, not less demanding. Its draft guidance asks for evidence commensurate with model risk rather than ruling out technologies. The first cGMP warning letter citing AI, to Purolea Cosmetics Lab on 2 April 2026, made the principle plain: “If you use AI as an aid in document creation, you must review the AI generated documents to ensure they were accurate and actually compliant with CGMP.” The failure was the missing review, not the AI.

For the Western market, Annex 22 is shaping up as the strictest standard. That makes it the safer baseline to design for.

3. Annex 22 key principles

The first section of the Annex clarifies the major elements: where it applies, and what kind of AI is allowed.

Is the application critical? The annex applies “where Artificial Intelligence models are used in critical applications with direct impact on patient safety, product quality or data integrity, e.g. to predict or classify data”. Criticality attaches to the application, not the model: the same extraction model is critical when reading a Batch Record whose values feed release, and non-critical when summarising an internal report.

For such applications, two principles apply:

1. Static models: The kind of models that are allowed: "models that do not adapt their performance during use by incorporating new data". Dynamic models that “continuously and automatically learn and adapt performance during use” should not be used in critical GMP applications. Retraining is not forbidden; learning in production is. A retrained model is a new version and a controlled change under §10.1.
2. Deterministic outputs and no LLMs: Models that, “when given identical inputs, might not provide identical outputs” should not be used in critical GMP applications. The draft names GenAI and LLMs explicitly.

The Annex clarifies that LLMs remain available for non-critical applications, provided "personnel with adequate qualification and training should always be responsible for ensuring that the outputs from such models are suitable for the intended use, i.e. a human-in-the-loop (HITL)".

One clause in the scope section is easy to miss and worth paying attention to: "Models may consist of several individual models, each automating specific process steps in GMP." Ensembles of specialised models are explicitly foreseen. The unit of validation is each model against the step it automates, not the product as a single black box.

4. Implications: which type of AI can be used, and where GenAI still fits

Some cases are easy to categorise, and quite a few fall in the grey zone. This is where implementation choices and overall processes will come into the picture. Because the ultimate test is: if the model fails, is patient safety at risk? Is data integrity compromised?

For grey-zone and non-critical cases, the quality risk assessment will be an important fall-back when scrutiny happens: keep a detailed record of the rationale and responsible person.

The Purolea warning letter shows what the non-critical column looks like without the human step. AI agents generated specifications, procedures and master production and control records, and nobody verified them. The task was not the violation. The missing review and approval was.

5. The Annex explained clause by clause

One concept worth emphasising here: human-in-the-loop is a trade-off with depth of testing and model performance. The less a model is tested, the more of its output a person must check, up to every output.

§10.5: “Depending on the criticality of the process and the level of testing of the model, this may imply a consistent review and/or test of every output from the model, according to a procedure.”

Practical recommendations for safe AI deployment

Besides regulatory steps and administrative documentation, there are various ways to make AI safer to use and easier to check.

For sensitive but non-critical use cases, here are some solid measures:

• Ground: feed LLMs with validated, structured data to foster better answers
• Constrain: for document drafting, use fixed templates with built-in rules
• Reference: every statement needs to keep a trace to the source, ideally showing immediately on a split screen view to facilitate effective checks (not fake HITL)
• Attribute: log model, prompt and parameters per output
• Responsibility of the setup: who approved what, when and why
• Break it down: lengthy processes should involve multiple HITL along the chain, to avoid the black box effect and untraceable errors, or said otherwise “don’t overdo it”.
• Guardrail the confidence: uncertainty or lack of source leads to an “n/a” answer, the case goes straight to an SME

Implications and actions you can take now to build great AI foundations

Whilst the FDA takes a risk- and responsibility-based approach, the EMA is much more restrictive (no LLMs) and much more demanding on documentation and testing procedure. In the final version of Annex 22, the EMA may soften the stance, but it’s unlikely to significantly alleviate the validation burden and the requirements on deterministic or highly reliable outcomes.

The requirements of Annex 22 also go beyond the validation process itself and impact the whole AI value chain: baseline performance of the pre-AI process, details on training sets, independence of training and test data, independence of staff, which all need to be thought through at the beginning of each AI project. For those building AI applications internally, pay attention to the infrastructure required to provide traceable model training, testing and monitoring.

The question is no longer “what can I do with AI” or even “can I trust AI”, but rather “how to build strong AI foundations across use-cases that will withstand scrutiny and evolve over time”. For us, that means:

• Starting now and not waiting for regulators to draw up the next paper
• Building for the GMP intent: risk-based, with AI choices adequate to the risk, likely regulation and consequences of model failures
• Finding the right compromise between doing it right and not drowning teams in lengthy administrative documentation
• Building AI validation as an organisational skill that enables safe and reasonably fast deployments
• Consolidating providers of AI systems and building with them an AI validation playbook that creates economies of scale across model versions, sites and use-cases

Editors’ note: this article was written by a human, with support from AI for drafting and visualisations