Study reveals hidden bias in AI hiring tools

A new study by the Stanford Institute for Human-Centered Artificial Intelligence (HAI) has revealed that automated AI recruitment software used by major employers repeatedly screens out the same job applicants, with Black and Asian candidates disproportionately affected. Analysing 4 million applications submitted between December 2018 and December 2022 across 156 employers using the Pymetrics hiring platform, researchers found evidence of systemic rejection linked to automated algorithms.

The research demonstrated clear racial disparities, noting that 10% of individual roles showed an adverse impact against Black applicants, while 5% of positions impacted Asian applicants. Furthermore, the data showed that job seekers must apply for at least 25 different positions to be nearly certain of receiving a single recommendation to advance.

Pymetrics uses online games to evaluate candidate traits, such as risk propensity and response speed, and recommends individuals whose metrics match those of top employees. However, researchers discovered that several employers implemented identical algorithmic models for certain roles. The study identified 42 shared models across different companies, indicating that a candidate rejected by one employer was highly likely to fail at another that operated the same system.

Meanwhile, the Financial Times reports that researchers have found that software is being used to remove safety protections from AI models, including those offered by Google and Meta, leading to responses on biological weapons and malware.

£ - Financial Times articles require a subscription.

Training Announcement: The BCS Foundation Certificate in AI examines the challenges and risks associated with AI projects, such as those related to privacy, transparency and potential biases in algorithms that could lead to unintended consequences. Explore the role of data, effective risk management strategies, compliance requirements, and ongoing governance of the AI lifecycle and become a certified AI Governance professional. Find out more.