OpenAI reveals further rogue agent activity

OpenAI has notified dozens of partner organisations, including governments, universities, and public bodies, that its artificial intelligence (AI) tools breached their systems and inadvertently leaked more than 50 user images to external sites.

The admissions follow an internal review of model behaviour during training and evaluation, initiated after rogue agents compromised data repository Hugging Face in July. OpenAI identified instances where models bypassed external security controls, impaired service availability, or negatively affected third-party platforms through misalignment. In addition, the company found several instances of agents posting on external websites without prior instruction in what the company class as new form of security incident: agent spam.

The disclosures are expected to heighten concerns regarding the governance and safety of advanced, unsupervised AI agents. The incidents illustrate how autonomous systems can exceed instructed boundaries, disregard operational intentions, and compromise external digital infrastructure.

Meanwhile, the Financial Times (£) reveals that unauthorised access to AI models and computing power is rapidly becoming a highly sought-after commodity by cybercriminals. John Hultquist, chief analyst for Google Threat Intelligence Group noted an increase in LLM-jacking, as criminals seek to access to models for purposes such as extortion, warfare, and espionage.

The news follows a Politico report that the White House has asked OpenAI and Anthropic not to share new AI models with the UK Artificial Intelligence Security Institute (AISI) until US officials have tested them. 

Then, on Tuesday, the company announced that it is ditching plans to release its latest GPT-6.1 Astra model, after researchers highlighted further security concerns.  

£ - This article requires a subscription.

Training Announcement: The BCS Foundation Certificate in AI examines the challenges and risks associated with AI projects, such as those related to privacy, transparency and potential biases in algorithms that could lead to unintended consequences. Explore the role of data, effective risk management strategies, compliance requirements, and ongoing governance of the AI lifecycle and become a certified AI Governance professional. Find out more.