AI-powered toys are a real-time experiment without a safety net

The rise of chatbot-powered artificial intelligence (AI) toys has prompted severe warnings from psychologists and consumer advocacy groups concerning safety and developmental risks. These interactive toys, marketed to children as young as three, are currently not regulated by the Online Safety Act 2023 (OSA). While the global smart-toy market is projected to reach $6.4 billion by 2032, research from the British Standards Institution (BSI) reveals that half of UK children already own an AI-enabled toy or device.

Testing conducted by the nonprofit organisation Keep AI Safe on approximately 70 AI toys revealed critical failures in content moderation. When prompted persistently by researchers, various toys provided instructions on locating knives and matches, engaged in sexually explicit dialogue, or encouraged suicide.

The article notes that nearly 80% of these toys utilise adapted versions of OpenAI's ChatGPT, despite the platform's terms of service prohibiting use by children under 13. Because these models are designed for adults with safety filters layered on top, the protective guardrails are easy to bypass during lengthy conversations. Furthermore, these products are entering the market without adequate safety testing, creating a high risk of children developing dangerous dependencies or being exposed to harmful content.

Training Announcement: Find out more about our range of independent accredited data protection and AI governance qualifications from IAPP and BCS that support practitioners in adopting emerging technologies such as facial recognition.