MIT develops 100% effective auditing technique to test Gen AI models for CSAM

Researchers from MIT, in collaboration with the child safety non-profit Thorn, have developed a new auditing technique to identify whether generative artificial intelligence (Gen AI) models have been adapted to produce illegal content, including child sexual abuse material (CSAM) and non-consensual sexual images.

Developed by Associate Professor Ashia Wilson, graduate student Vinith Suriyakumar, and researchers from MIT's Healthy ML Lab, the method evaluates open-source Gen AI models without prompting them to create illegal outputs. Instead of producing harmful imagery, the technique analyses the inner workings of the models by examining hidden representations to determine whether they have been specialised to produce harmful content.

During testing, the auditing procedure identified model variations specialised to produce illegal material with 100% accuracy. The development provides hosting platforms, law enforcement and regulators with a tool to test models for malicious capabilities, enabling platforms to flag and remove unsafe models or block them from being uploaded. 

The research was presented at the International Conference on Machine Learning's Trustworthy AI for Good workshop.

Training Announcement: Freevacy offers a range of independently recognised professional AI governance qualifications and AI Literacy short courses that enable specialist teams to implement robust oversight, benchmark AI governance maturity, and establish a responsible-by-design approach across the entire AI lifecycle. Find out more.