The EU AI Act Is Here — What It Means for Clinical Trials

Contributing Expert: Ian Davison, Subject Matter Expert at Medrio

The European Union’s (EU) Artificial Intelligence (AI) Act touches many industries, including clinical research. The EU AI Act aims to comprehensively regulate AI systems across industries in the European Union. 

The goal of the EU AI Act is to provide governance while building trust and supporting the ethical use of AI. It aims to support innovation while minimizing risk. 

In this article, you’ll learn more about how the EU AI Act will impact clinical research. We’ll also explore where various clinical trial activities may fall within the Act’s risk classification system.

How is the EU AI Act Affected by Other EMA Guidance?

Along with the EU AI Act, sponsors still need to comply with any relevant guidance provided by the EMA. Point 64 of the Act highlights that medical devices using an AI system may pose risks not specifically addressed within the Act. Therefore, “this calls for a simultaneous and complementary application of the various legislative acts.”

Sponsors have some flexibility in ensuring compliance. For example, they may integrate necessary testing, reporting processes, and documentation required under the Act into existing documentation and procedures required by the EMA.

Wondering about the current state of AI? Download our AI in Clinical Trials eBook to find out more. 

When Will the EU AI Act Take Effect?

The EU AI Act will gradually take effect. It will apply two years after entry into force on August 2, 2026.

Important dates to know:

• February 2, 2025: All prohibitions, definitions, and provisions related to AI literacy applied on this date.
• August 2, 2025: The rules on governance and the obligations for general-purpose AI will apply on this date.
• August 2, 2026: The AI Act will apply two years after entry into force on this date.
• August 2, 2027: The obligations for high-risk AI systems that are classified as high-risk because they are embedded in regulated products, listed in Annex II (list of Union harmonization legislation), come into effect on this date.

Looking for more information about upcoming dates? Check out the European Question and Answers webpage.

Risk Classification Within the EU AI Act for Clinical Research

The EU AI Act takes a risk-proportionate approach. This approach includes four levels of AI systems, ranging from minimal to unacceptable. Each risk level has corresponding regulatory requirements. 

The EU AI Act categorizes AI systems into four risk levels:

• Unacceptable risk: AI systems that pose a clear threat to fundamental rights, such as real-time biometric surveillance, are banned.
• High risk: AI in critical areas, like healthcare, recruitment, and law enforcement, faces strict compliance requirements, including transparency, data governance, and human oversight.
• Limited risk: AI systems with potential for manipulation must disclose their use of AI to users.

Minimal risk: Many AI systems, such as recommendation algorithms and spam filters, face no specific regulations beyond existing laws.

How are clinical trials activities classified?

“High risk” is the highest acceptable risk level and may be subject to a more stringent set of requirements. Several clinical trial-related AI systems could fall into this category.

AI systems used in clinical trials will likely be considered “high risk” if they are part of:

• Patient recruitment
• Allocation of treatment
• Diagnostics
• Data management
• Synthetic data generation
• Decision-making 
• Medical devices (Point 50 in EU AI Act)

What are the requirements for high-risk AI systems?

High-risk AI systems must adhere to rigorous standards for data quality, transparency, and human oversight. These standards include comprehensive documentation and conformity assessments to verify compliance.

To meet EU AI Act’s mandatory requirements for trustworthiness, sponsors should consider: 

• Data quality
• Documentation and traceability
• Transparency
• Human oversight
• Accuracy
• Cybersecurity and robustness

How the EU AI Act Impacts Data Management in Clinical Trials

The EU AI Act's requirements may influence how people use AI in clinical trials, particularly data collection and analysis. When considering how to meet requirements, sponsors should focus on data transparency, data governance, and human oversight.

Ensuring that AI systems meet standards will be important for clinical trial integrity. Sponsors can do this through limited use of blackbox AI and active bias mitigation.

Limited use of black-box AI 

The Act pushes for explainable AI. Therefore, AI models must provide reasoning behind their outputs. This requirement could challenge certain deep-learning models unless they offer sufficient transparency.

Bias mitigation

AI systems that generate or analyze trial data must meet stringent accuracy and fairness requirements. Bias mitigation will be key. It ensures AI doesn't inadvertently skew patient selection or data interpretation.

In a Q&A, lawmakers stated that AI systems must “not produce biased results, such as false positives or negatives, that disproportionately affect marginalised groups, including those based on racial or ethnic origin, sex, age, and other protected characteristics.” 

When using AI systems in clinical trials, sponsors should help avoid bias by:

• Training and testing models with “sufficiently representative datasets”
• Ensuring systems are traceable and auditable
• Keeping appropriate documentation, including the data used to train the algorithm 
• Monitoring the system regularly
• Addressing potential risks promptly

Prepare for the EU AI Act Today

Sponsors and CROs must understand and comply with the EU AI Act. Companies that proactively align their AI systems with these regulations may gain a competitive advantage in the EU market. Focusing on compliant, scalable AI solutions may drive efficiency in research.

In preparation for the full implementation of the EU AI Act, take a proactive approach. 

To do so, focus on auditing AI systems, improving documentation, and implementing bias mitigation strategies. These activities should already be familiar to the developers of clinical research systems since computer system validation has long been rigorously regulated.

How are today’s trials using AI? Learn more in our State of AI in Clinical Trials eBook.

The edition of “Trial Talks” does not constitute legal or other professional guidance. Please refer to proper industry documentation or known notified bodies for legal or professional advice.