Who exactly has the power to shut an errant AI system down

An article by Joseph Wallace, Director of Data and AI Governance at Adobe, in MIT Sloan Management Review highlights the difference between actual artificial intelligence (AI) governance and merely implementing tools to monitor AI model performance. 

The author asks an important question: who has the authority to shut down an AI model found to be doing something it shouldn't? 

Wallace argues that if you cannot answer this question, your AI governance programme is merely performative. 

The article outlines how corporate AI governance often lacks actionable authority, despite widespread claims of oversight by business leaders. Over recent years, an industry has built infrastructure for visibility, such as registries, dashboards, and risk frameworks. However, these tools identify problems but do not provide mechanisms to solve them. While advisory ethics officers and councils can flag issues, they generally lack the power to deactivate a model. Instead, ultimate decision-making typically rests with revenue-focused product teams incentivised to prioritise shipping deadlines over risk containment.

Instead, Wallace counters that true governance requires separating the authority to stop or pause an AI model from the teams that benefit from deploying it. This structural independence is increasingly urgent as enterprises deploy hundreds of unmonitored systems across the organisation. Rather than slowing down adoption, organisations must treat structural design with the same importance as technology. Wallace concludes that effective AI governance frameworks should clearly identify specific, named individuals who have the organisational standing, job security, and explicit responsibility to shut down a harmful AI system, ensuring they can exercise that authority even when it conflicts with commercial roadmaps.

Training Announcement: Freevacy offers a range of independently recognised professional AI governance qualifications and AI Literacy short courses that enable specialist teams to implement robust oversight, benchmark AI governance maturity, and establish a responsible-by-design approach across the entire AI lifecycle. Find out more.