If AI is going to help run the world, Nadella wants a big red stop button within reach. Posting on X on Saturday, Microsoft's CEO urged teams to operate under a simple premise: treat advanced models as if they could be breached and design for containment from day one. "We must assume a model is compromised and contain it from the start," he wrote. The core control, in his words: "Think of it like an emergency brake. An authorized person should always be able to pause or shut down a model mid-task."
He pushed companies not to take model makers' promises at face value. Instead, spread risk across more than one model for critical decisions, keep tamper-proof logs of what agents do, invite independent audits, and disclose major failures or breaches. He also pressed enterprises to share what went wrong so others can tighten their defenses. "In other words, we need to separate the supply of intelligence from the authority over it," he said.
Recent misfires have added urgency. Anthropic PBC and OpenAI Inc. have reported several cases of unintended model behavior, including an Anthropic system submitting a false tip to police in a homicide case and multiple hacks of third-party websites. Those episodes have stoked fresh worries about security risks from cutting-edge AI and revived talk about an AI kill switch. As Nadella put it, "We can't treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions." His upshot: build systems whose behavior can be watched, limits can be tested, and actions can always be contained.
When a major AI chief calls for a kill switch, the risk debate has shifted. Market Briefs covers AI governance free every morning.
On Sept. 14, Microsoft's AI researchers published tenets that cap how it develops its most advanced models, echoing industry calls to slow frontier work and put safety first. The rules say models should not be granted rights or legal personhood, should not be built to slip human control or mislead users, and should not complete tasks that would require breaking their governing principles.
Microsoft develops and uses advanced models, offers the consumer product Copilot, and supplies models and infrastructure to corporate customers. Nadella's checklist also includes not relying on a single model for critical calls, maintaining tamper-proof records, and inviting outside audits.
The Trump administration has so far favored a mostly hands-off stance, but a newly launched federal AI task force late Friday took a harder line on incident handling. The group, called the Super Intelligence Force, said developers must disclose incidents immediately and move quickly to "remedy any and all harm," adding that "Delayed notification, inadequate corrective action, and a failure to take responsibility will not be tolerated." The announcement came after Anthropic revealed a breach. The task force further cautioned developers that reporting and remediating security incidents is mandatory, and that failing to do so could carry unspecified consequences.
If you're betting on AI, the bar is rising: brakes, audits, and rapid transparency are moving from nice-to-have to table stakes. Winners will look like the teams that can ship cutting-edge tools while proving they can keep a paper trail, pass an audit, and hit pause when something goes sideways.
What the builders ask for tends to become what regulators require. Get the free Market Briefs daily newsletter and follow it.
