43 Accepted

Mandate human review and verification of AI-assisted decisions affecting individuals

Conclusion
The potential for human error and bias notwithstanding, deployers should not solely rely on outputs produced with AI tools to determine their decision-making, particularly in areas that could affect the rights and standing of the individuals or entities concerned, such as insurance decisions or recruitment. These algorithmic decisions should always be reviewed and verified by trained humans, and those affected should have the right to challenge these decisions—a process that should also be human-centred.
Government Response Summary
The government states that existing UK data protection law already provides safeguards for individuals, including the right to human review and to contest automated decisions. It also notes the Algorithmic Transparency Recording Standard requires publishing information on human review and the Generative AI framework recommends human involvement.
Paragraph Reference
161
Government Response
Accepted
HM Government Accepted
AI-driven automated decision-making can play an important role in delivering effective services and improving productivity. The UK’s data protection framework strikes a balance between enabling the best use of the technology whilst providing safeguards for individuals when they matter most. Individuals have rights under the UK’s data protection law, where they have been subject to automated decision-making without human involvement, including profiling. These rights apply to all forms of automated decision- making regardless of the technology being used. Where solely automated decisions are based on the use of personal data and have a legal or a similarly significant effect on individuals, data protection law requires organisations to offer a range of safeguards to individuals. These safeguards include informing them of the automated decision, enabling them to contest the decision, and the right to obtain human review. In addition to these safeguards, data protection principles of accuracy, transparency, purpose limitation, data minimisation, confidentiality, and accountability apply to all processing of personal data, including where forms of automated decision-making is used. Within the public sector specifically, the Algorithmic Transparency Recording Standard (ATRS) requires Government departments, and in time the broader public sector, to proactively publish information about how and why algorithmic tools are embedded in broader decision-making processes. This includes explanations of when and where human review takes place. In the Generative AI framework for HMG, we recommend maintaining appropriate human involvement in automated processes, and that developers and users of AI tools ensure that there is a human-in-the-loop who can oversee outputs when generative AI is in use in situations with a high impact. Government response to Committee recommendation 44
Addressee Bodies
Department for Science, Innovation and Technology
Timeline
Recommendation age 2.0 yrs
Report published 28 May 2024