AI Regulation for Medical Devices: EU AI Act, MDR, and FDA §524B

AI regulation for medical devices is among the most complex compliance landscapes in the world. According to the European Commission, medical AI systems fall simultaneously under the EU AI Act as high-risk applications and under the Medical Device Regulation (MDR 2017/745) as Software as a Medical Device (SaMD). In the United States, FDA §524B of the FD&C Act adds a third compliance axis. The critical gap across all three frameworks: none requires genuinely independent pre-deployment bias auditing.

How does the EU AI Act classify medical AI systems as high-risk?

The EU AI Act Annex III classifies AI systems used in medical diagnosis, treatment recommendations, and patient monitoring as high-risk applications. This triggers obligations under Articles 10, 13, 14, and 16: training data governance, transparency requirements, human oversight mechanisms, and accuracy/robustness/cybersecurity standards. Article 10 specifically requires that training, validation, and testing datasets be subject to data governance practices that identify and address known or foreseeable biases. The critical word is 'known' — which means the obligation applies only to biases the developer is already aware of, not to latent biases discoverable only through independent technical audit. This is the enforcement gap.

What does the MDR conformity regime require for Software as a Medical Device?

Medical Device Regulation 2017/745 classifies Software as a Medical Device (SaMD) by risk level. AI-powered diagnostic tools that influence clinical decisions fall in Class IIa or above, requiring Notified Body conformity assessment. The MDR focuses on clinical evaluation, performance testing, and post-market surveillance — not on algorithmic bias specifically. This creates a dual compliance gap: a medical AI system can achieve MDR conformity and EU AI Act compliance while still containing discriminatory patterns in its training data that were not subject to independent audit. The ISO quality technical standard and the RGPD legal accuracy standard address different dimensions of the same system, without a binding mechanism that bridges them.

How does the FDA §524B requirement differ from European medical AI regulation?

FDA §524B of the Federal Food, Drug, and Cosmetic Act requires manufacturers of cyber medical devices — including AI-enabled medical software — to have a plan for monitoring, identifying, and addressing cybersecurity vulnerabilities post-market. It also requires a bill of materials for software components. The regulation focuses on cybersecurity rather than algorithmic bias as such. This means a medical AI system in the United States can comply with §524B while systematically producing less accurate diagnoses for minority patient populations — a form of algorithmic bias with documented clinical consequences. The framework I propose in Humanoide en la Torre de Babel requires a pre-deployment Neural Analysis audit specifically for bias in training data and hidden layers, independent of cybersecurity compliance.

What are the specific bias risks in medical AI systems that current regulation does not address?

Medical AI bias takes several documented forms. Training datasets for diagnostic AI systems are disproportionately composed of data from high-income, predominantly white patient populations. This produces systems that are less accurate in diagnosing conditions in darker-skinned patients, in women (particularly for cardiovascular disease), and in populations from low-income countries. The consequences are not statistical abstractions — they are misdiagnoses, delayed treatments, and preventable deaths. Neural Analysis addresses this through mandatory pre-deployment demographic disaggregation testing: the system must demonstrate equivalent diagnostic accuracy across demographic groups before receiving deployment authorization, not after clinical harm has been documented.

What would a genuinely independent pre-deployment audit for medical AI look like?

A genuinely independent pre-deployment audit for medical AI requires three elements absent from all current frameworks. First, the auditing entity must have no financial relationship with the device manufacturer or the healthcare institution deploying the system. Second, the audit must use technical interpretability methods — saliency maps, TCAV, perturbation stress testing — to examine the hidden layers of the model for demographic bias, not just evaluate aggregate performance metrics. Third, the audit must have legal veto power: if the system fails the bias audit, it cannot be deployed until remediation is documented and re-audited. This is the Neurological Birth Certificate standard. Current FDA, MDR, and EU AI Act conformity processes meet none of these three requirements simultaneously.