Ofcom opens X sexual imagery investigation, concerns emerge over Musk's concessions

The public outcry over the non-consensual sexualised images generated by the Grok AI chatbot on X has continued after the platform confirmed that image generation and editing had been restricted to paying subscribers.

In a statement on Friday, Secretary of State for Science, Innovation and Technology, Liz Kendall, said: "Sexually manipulating images of women and children is despicable and abhorrent. It is an insult and totally unacceptable for Grok to still allow this if you're willing to pay for it. I expect Ofcom to use the full legal powers Parliament has given them." She went on to say that she expected the regulator to investigate the next steps "in days not weeks."

Downing Street is also unhappy with the change to restrict access to paying users only. In a statement given to The Guardian, a spokesperson said: "The move simply turns an AI feature that allows the creation of unlawful images into a premium service."

Ofcom responded accordingly, confirming it would accelerate its investigation into X. This was followed by a typical reaction from Elon Musk, accusing the UK government of seeking to suppress free speech. 

In Europe, the Italian data protection authority (DPA) issued a warning to AI providers to ensure their services comply with data protection laws and regulations. The DPA also confirmed that it is working with the Irish Data Protection Commission (DPC) on potential further enforcement activities.

This was followed by a warning from Henna Virkkunen, Executive Vice President of the European Commission for Technological Sovereignty, Security and Democracy, telling Elon Musk to fix Grok quickly or face consequences under the Digital Services Act (DSA). 

As the standoff continued to escalate on Monday, the UK Secretary of State for Business and Trade, Peter Kyle, told Sky News: "Let me be really clear... X is not doing enough to keep its customers safe." He added that the government would fully support any action taken by Ofcom against X, including a possible ban in the UK. 

Later on Monday, Ofcom released details about the formal investigation it has opened into X under the Online Safety Act 2023 (OSA), clarifying that it aims to determine whether X has complied with its duties to protect users from illegal content and to safeguard children from harmful content.

The investigation seeks to identify whether X has taken the necessary steps required under the OSA to assess the risk and implement appropriate measures to prevent individuals in the UK from encountering or accessing illegal content, including non-consensual intimate images and child sexual abuse material (CSAM). Where such content exists on the platform, Ofcom requires assurances that X will remove illegal content as soon as it becomes aware of it, ensure user privacy, and comply with relevant UK data protection laws. In addition, the investigation will determine whether X is evaluating the potential risks posed by its services to UK children and whether it conducts regular risk assessments before making any significant changes or updates. Finally, Ofcom will assess the processes and measures X has implemented to provide effective age verification methods to protect UK children from viewing pornography.  

Given the level of coverage the image editing scandal has received in the UK and Europe, it is worth noting that it is also making waves across the Atlantic. Last Friday, three US senators wrote to Google and Apple, asking them to remove Grok and X apps from their respective app stores, arguing that the spread of non-consensual sexualised images violates app store rules.

The US government, however, took a different line, standing by X and warning that "nothing is off the table when it comes to free speech" if the UK government follows through on its threat to ban the social media platform. Speaking to GB News, US Under Secretary of State Sarah B. Rogers, said: "Let's wait and see what Ofcom does and we'll see what America does in response." 

Then, on Wednesday, the California attorney general's office announced that it had also opened an investigation into X. In a statement, AG Rob Bonata said: "As the top law enforcement official of California tasked with protecting our residents, I am deeply concerned with this development in AI and will use all the tools at my disposal to keep California’s residents safe." 

At this point, the pressure on X had become unsustainable, prompting what appeared to be a significant climbdown.

During Prime Minister's Questions (PMQs), Sir Keir Starmer welcomed reports that X is taking steps to comply with UK law. The Prime Minister noted that while X is working to ensure legal compliance, the government is prepared to take further measures if necessary and will continue to monitor the situation. Starmer confirmed that the government is ready to strengthen existing legislation or introduce new laws to prevent online abuse. He also clarified that authorities will not back down in their efforts to hold social media companies accountable for harmful content and that Ofcom's independent investigation into X will proceed. 

By Wednesday evening, X had posted a statement on its @Saftey account on X confirming that: "We remain committed to making X a safe platform for everyone and continue to have zero tolerance for any forms of child sexual exploitation, non-consensual nudity, and unwanted sexual content." 

The statement goes on to explain that X has implemented geoblocking in jurisdictions where generating images of real individuals in underwear or similar attire is illegal. The company stated that it continues to remove high-priority violative content and will report accounts involved in child exploitation to law enforcement. These updates aim to prevent misuse of generative AI technologies and ensure that users who violate platform policies or national laws are held accountable.

As the temperature cools on the latest social media outrage (even if only temporarily), concerns are growing over the concentrated power of technology platforms. The Open Rights Group (ORG) argues that proactive regulation could have prevented the latest harms associated with X. In a blog article, ORG highlights that while the EU Artificial Intelligence Act (AI Act) will require providers to identify and mitigate risks, the UK government has delayed its own AI legislation under pressure from US providers. ORG goes on to stress that the lack of a comprehensive, risk-based framework is a predictable failure and calls on the government to introduce an AI Bill that bans harmful technologies. 

Taking a wider view of the scandal, experts warn that the use of AI to harm women is only just beginning, despite attempts to curb the technology. While X has said it would introduce safeguards to prevent the spread of non-consensual sexualised imagery, enthusiasts on online platforms continue to exploit alternative tools with fewer restrictions. The rapid development and accessibility of these technologies present a significant challenge for global regulators.

According to The Guardian article, one user of Grok's image editor said: “Since discovering Grok AI, regular porn doesn’t do it for me anymore, it just sounds absurd now,” while another agreed, “If I want a really specific person, yes.”

Notwithstanding the UK government's moves to criminalise the creation of non-consensual intimate images, the concern is that the ease of distribution makes enforcement almost impossible. The high demand for specific, AI-generated sexual content highlights a growing systemic problem that current legislation may struggle to contain as the technology continues to evolve and proliferate globally.

On Friday, and despite the restrictions announced this week and its assurances to crack down on misuse, The Guardian has discovered that X continues to allow the posting of non-consensual content via its standalone app. 

[Sigh...]

Training Announcement: The BCS Foundation Certificate in AI examines the challenges and risks associated with AI projects, such as those related to privacy, transparency and potential biases in algorithms that could lead to unintended consequences. Explore the role of data, effective risk management strategies, compliance requirements, and ongoing governance of the AI lifecycle and become a certified AI Governance professional. Find out more.