Treating AI Agents like employees undermines human accountability

New research by Boston Consulting Group and Boston University suggests that treating artificial intelligence (AI) as an employee rather than a tool can create significant organisational risks. People often attribute human traits to non-human things. However, while leaders may like to anthropomorphise agentic AI systems to signal ambition or reduce worker apprehension, a randomised experiment indicates that humanising the technology shifts accountability away from individuals. This approach can also lead to increased escalation of issues, diminished review quality, and the erosion of professional identity and trust.

In addition, the study found that treating AI as a colleague does not meaningfully increase a workforce's intention to adopt or integrate the technology. Instead, it creates costly complications that may outweigh the benefits of streamlined work. The findings highlight that while agentic AI has the potential to transform organisational capabilities, the primary challenge lies in integration. Successful adoption requires preserving human accountability and maintaining work quality, ensuring that employees can work effectively alongside autonomous systems without compromising professional standards or the accuracy of outputs.

Training Announcement: The BCS Foundation Certificate in AI examines the challenges and risks associated with AI projects, such as those related to privacy, transparency and potential biases in algorithms that could lead to unintended consequences. Explore the role of data, effective risk management strategies, compliance requirements, and ongoing governance of the AI lifecycle and become a certified AI Governance professional. Find out more.