How should bias be detected and corrected in high-risk systems?

Where can bias enter a high-risk system?

Bias can enter through historical patterns in training data, through labelling choices, through under-representation of affected groups, through proxy variables that correlate with protected characteristics, and through deployment conditions that differ from testing conditions. It can also enter through the definition of the task itself, where the outcome the system is optimised for reflects an existing inequality. A bias examination should therefore look at the data, the labelling, the model, the evaluation, and the deployment context together, rather than treating bias as a property of a single dataset. The examination should record the sources considered, the methods used, and the limitations of each method, so that a finding of no bias is understood as a finding within the limits of the examination, not as proof that no bias exists.

How should performance differences be measured?

Performance differences should be measured by comparing relevant performance measures across affected groups, using data and methods that are appropriate to the task and the groups concerned. The measures chosen should reflect the consequences of error for each group, not only overall accuracy, because a system can perform well on average while performing poorly for a group that is already disadvantaged. Measurement should be documented with the groups compared, the measures used, the data relied on, and the thresholds or criteria applied to judge a difference as material. Where direct group data are unavailable or inappropriate, the provider should record what alternative methods were used, what they can and cannot show, and what uncertainty remains. A measurement that is not documented cannot be relied on or challenged by a reviewer.

For the broader regulatory framework, see High-risk requirements overview.

What correction measures are available?

Correction measures depend on the source of the bias found. They can include changes to data collection or preparation, rebalancing or augmenting datasets, changes to labelling, changes to model design or thresholds, changes to the deployment workflow, additional human oversight, or limitations on use where a bias cannot be corrected for a particular context. Each measure should be tested to check that it reduces the bias without creating a new harm, and the test should be recorded. A correction that improves one measure while worsening another should be assessed as a trade-off, with the reasoning recorded and the residual risk addressed in the risk management file. Correction is an iterative process: a measure should be monitored after deployment to check that the improvement holds in real conditions.

When can special category data be used for bias work?

Special categories of personal data may be processed exceptionally for bias detection and correction only where Article 4a permits it, to the extent strictly necessary, and subject to all the cumulative conditions and safeguards in that article, together with data protection law. For providers of high-risk systems, Article 4a(1) applies in accordance with Article 10(2)(f) and (g). For other providers and deployers, Article 4a(2) applies where strictly necessary to address biases likely to affect health and safety, fundamental rights, or to lead to prohibited discrimination. In every case, the provider or deployer must first consider whether the objective can be effectively fulfilled by other data, including synthetic or anonymised data, must apply technical limits on reuse, state-of-the-art security and privacy-preserving measures including pseudonymisation, strict access controls and documentation, must not transmit the data to other parties, and must delete the data once the bias has been corrected or the retention period ends, with the processing record stating why the processing was strictly necessary. The necessity assessment and the safeguards should be documented before processing begins, not reconstructed afterwards.

How should bias work be kept live through the lifecycle?

Bias examination should be repeated when data, populations, deployment contexts, or the system itself change, and should be linked to post-market monitoring so that performance differences in operation are detected. The provider should set review triggers, assign an owner, and record each examination, its findings, the measures taken, and the residual concerns. Deployers should feed back operational observations, including complaints and patterns in outcomes, through the agreed channel, and providers should record what was received and what followed. A bias file that is produced once before release and never revisited will miss drift, population change, and new deployment contexts that can create or worsen bias after release. The file should therefore be treated as a living record, with dated entries for each review.

Continue with Article 10 data and data governance.

What does this guide not decide?

This guide does not decide whether a particular system is biased, whether a bias measure is sufficient, or whether Article 4a conditions are met for a specific processing operation. It does not set a fairness metric, a threshold for a material difference, or a data protection law basis. Those questions depend on the data, the task, the affected groups, the safeguards in place, and the consolidated legal text together with data protection and equality law. A provider or deployer should confirm its bias and Article 4a position with a competent authority or legal adviser, document the necessity assessment before processing special category data, and keep the bias file under review.

Frequently asked questions

Continue through this AI Act cluster

• High-risk requirements overview
• Article 10 data and data governance
• Article 9 risk management for high-risk systems
• Testing and validation for high-risk systems
• Provider responsibilities
• What technical documentation does Article 11 require?