Study maps disparity between AI providers and user confidentiality expectations

A new academic study: You Trust Your Chatbot With Everything. Should You? has mapped the discrepancy between user expectations of confidentiality and the actual data practices of major consumer artificial intelligence (AI) providers. The research analysed five leading chatbots: ChatGPT, Gemini, Claude, Grok, and DeepSeek, to evaluate how conversation data is managed for everyday users, who often disclose sensitive health, financial, and emotional information.

The study identified a landscape of structural opacity regarding how providers reuse data. Key findings indicate that conversations are frequently used for AI training and improvement, are subjected to human review by annotators, and are shared across operational and ecosystem channels. There are also risks related to monetisation through advertising and data spillover.

The researchers propose several safeguards to address these privacy risks, including decoupling chat history from AI training and implementing a sealed mode for high-stakes topics. They also recommend standardised disclosures when humans review conversations and a regulatory-first approach to conversational advertising. While enterprise versions often provide better protections, the study warns that standard consumer interfaces invite an intimacy that is not supported by professional secrecy or clear technical boundaries. Ultimately, the findings suggest that current opt-out mechanisms and transparency measures remain insufficient for protecting personal privacy.

Training Announcement: Find out more about our range of independent accredited data protection and AI governance qualifications from IAPP and BCS.