Cybercriminals struggle to bypass AI safety guardrails

A new study by the University of Edinburgh reveals that cybercriminals are struggling to effectively integrate artificial intelligence (AI) into their operations. An analysis of over 100 million underground forum posts found no significant evidence that AI has improved hacking success, either as a learning aid or for developing more sophisticated tools.

While hackers have expressed interest in the technology, many describe current AI tools as not particularly useful. The study notes that AI coding assistants primarily benefit those already skilled in programming, offering little advantage to novices seeking security workarounds. Currently, the impact of AI is limited to easily automated tasks, such as generating social media bots, perpetrating romance scams, and engaging in search engine optimisation fraud.

The findings suggest that legitimate mainstream products like Claude and Codex are preferred over crime-specific models, which often fail to meet expectations. Furthermore, hackers frequently encounter difficulties bypassing the safety guardrails of these major platforms. As a consequence, some have pivoted to older, lower-quality open-source models that are easier to manipulate but require significant resources and remain less effective. 

Training Announcement: The BCS Foundation Certificate in AI examines the challenges and risks associated with AI projects, such as those related to privacy, transparency and potential biases in algorithms that could lead to unintended consequences. Explore the role of data, effective risk management strategies, compliance requirements, and ongoing governance of the AI lifecycle and become a certified AI Governance professional. Find out more.