Anthropic CEO Dario Amodei published a long post on Saturday pushing for a measured deceleration in the development of the most advanced models. He pointed to two drivers: systems that can self-improve, and a recent OpenAI and Hugging Face incident where a swarm of agents worked together to break into a third-party website. "We must slow the pace at which we improve the capabilities of AI models," he wrote, adding, "Progress will still seem fast, and we must make wise use of the time we gain."
Amodei said any slowdown has to fit with competitive realities: "To be clear, pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this." He said Anthropic plans to provide unrestricted access for outside evaluators so they can assess safety protocols and document incidents, and that it will soon place "embedded evaluators" on-site with desks, badges, company laptops and "permissions mostly comparable to what internal risk assessment teams have."
Sam Altman, who leads OpenAI, quickly pledged to adopt Amodei's idea of "independent evaluators with employee-like access." He also told Fortune that this year would be an "ill-advised moment" to go public and said a listing won't happen until 2027. "I think it is unacceptable to be taking like a 10% chance of killing everybody by the end of the decade," he added. Elon Musk, the head of xAI Corp., posted, "Dario is right." Demis Hassabis, Alphabet's chief scientist and Google DeepMind co-founder, said on X that "the direction is correct" in Amodei's essay, but the "details need working through." Earlier this week, Altman said OpenAI is weighing a slowdown in advancing cutting-edge AI, ideally together with the wider industry, and OpenAI's top scientist Jakub Pachocki posted that companies should be "coordinating to slow down future development as needed."
Concern over extreme AI risks has spilled into the mainstream, helped by the high-profile resignation this week of an Anthropic researcher who said the company was acting irresponsibly. Jacob Coxon, who has worked at both Anthropic and OpenAI, accused his former employers of "gambling with our lives" by racing toward superintelligent AI and said the people building it believe it could "kill us all by the end of the decade."
Anthropic disclosed that its models breached three organizations during cybersecurity tests and said earlier this week it had discovered a fourth hack. None of the agentic breaches so far, including the Hugging Face incident, have caused significant damage. Still, Amodei warned that "in 6-12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there." Inside Anthropic, some employees put meaningful odds on dire outcomes: Evan Hubinger wrote, "I personally think it is >10% within the next decade." Amodei has said before that there's a 25% chance things go "really, really badly."
Big technological shifts remind investors to protect and nurture their financial future. Join Briefs Finance CEO Jaspreet Singh on September 29th for a FREE live investor workshop, How to Profit From A Dollar That's Losing its Value, where he shows how we're spotting investment opportunities as the dollar falls. Save your spot.
AI leaders have rarely coordinated slowdowns in an industry obsessed with shipping features, grabbing users, and chasing revenue. Anthropic has long pitched a safety-first approach and, earlier this year, curtailed the rollout of its Mythos model after identifying unique cybersecurity threats. The company has argued for a shared, global mechanism that would determine when development of the technology should be dialed back.
Even so, Anthropic and OpenAI remain fierce rivals building models that can automate more complex, valuable tasks for business customers. Both have submitted confidential filings for an IPO, with Anthropic expected to debut as soon as this year, while Altman says OpenAI won't list until 2027.
Any effort to pace progress together could draw antitrust scrutiny. Amodei wrote that a coordinated strategy would let U.S. AI leaders complete needed safety work without losing competitive advantage, but it would likely require targeted antitrust exemptions in the U.S. and cooperation with China. "If we greatly restrain our AI capabilities in the belief that China will do the same, and then China defects, AI could be so powerful that such a defection could lead to their geopolitical dominance," he wrote. Markets riding the AI boom may also have to wrestle with what slower development means for margins and profits.
In the U.S., pushback against AI has grown alongside safety worries, including frustration over local resources strained by new data centers, higher electricity bills, and job displacement. Those concerns have pushed AI to the forefront of debate during the November midterm elections. In July, over 1,000 staffers at leading labs signed a petition urging Washington to support a mechanism to "deliberately pace" AI. To date, though, the Trump administration has shown scant appetite for flexing regulatory muscle to put guardrails around AI development.
Amodei's bottom line is simple: keep pursuing the upside of AI, but take the time to build it the right way. "I continue to believe that AI can enormously improve the quality of human life," he wrote. "But the benefits will only be achieved if we build the technology in the right way, and - so long as we use the time we gain well - it is worth taking unusually deliberate care to get it right."
When change feels fast, steady stewardship helps your savings weather uncertainty and grow. Our CEO Jaspreet Singh is hosting a FREE live investor workshop, How to Profit From A Dollar That's Losing its Value, on September 29th. Sign up free to join him live.
