Not long ago, outside AI evaluators barely registered in a market worth trillions. Now, with new models shipping at a rapid clip and money flooding in, a handful of independent groups are being asked to help police risks. These organizations, often nonprofits, test what models can do, probe for failure modes, and call out unsafe behavior.
Last month, Anthropic CEO Dario Amodei said the company would place independent evaluators inside its walls, and OpenAI's Sam Altman backed the idea. Still unresolved: how these groups will be funded, how much they can see, and where their reports will go.
Amodei's viral essay also urged a "slower pace" for cutting-edge model development after researchers left and raised alarms about existential risks. That push, combined with industry support, has put evaluators on a much faster track.
Anthropic and OpenAI are seeking help from compact outfits such as Model Evaluation and Threat Research (METR), Apollo Research, and Transluce. The evaluator scene today is mostly small specialists alongside big consultancies, with Accenture among the larger firms involved.
"I've worked in politics and policy for the last 20 years of my life, and I've never seen an issue move so fast on so many different political spectrums," said Andrew Freedman, who leads the policy nonprofit Fathom. He expects an "influx of capital" into the ecosystem.
There is already tension as labs make room for outsiders. OpenAI dismissed three employees last week for "violating our policies on accessing and handling sensitive company information," a spokesperson said. Two of them, Mikita Balesni and Tomek Korbak, said they believe their terminations were tied to how they communicated with third-party evaluators. "My former colleagues are telling me they are confused about what to believe," Balesni wrote on X.
OpenAI rejected that account. Anthropic did not comment.
Independent evaluation is becoming infrastructure for the whole AI industry. Market Briefs covers that build-out free every weekday.
Money and structure are the sticking points. "To a degree, the problem, as always, is money," said Suresh Venkatasubramanian of Brown University, where he is a computer science professor. At present, Anthropic, OpenAI, and the boom's infrastructure partners are, in effect, determining the ground rules.
Critics liken that to letting big banks police financial crises or allowing drugmakers to release treatments without regulators. Trump recently praised AI leaders for their "tremendous self-policing," signaled he doesn't plan to slow the sector's growth, and encouraged companies to work with outside evaluators through the voluntary accord.
Capacity is another hurdle.
The funding tide is turning, though. Vals AI CEO Rayan Krishnan said the for-profit evaluator grew its team this year from eight employees to about 30 and, in August, announced a $40 million round. METR, a nonprofit, said in August it secured about $71 million in commitments over six months, compared with total 2024 contributions of $13.6 million in its latest IRS filing. METR said it declined payment for that work.
The imbalance also raises conflict risks. "If you want true third-party evaluation, you need true independence financially and otherwise," Venkatasubramanian said.
Anthropic has started outlining a path. Given the urgency, Anthropic said it will fund Accenture's contributions directly. The company added there are no standards yet for what embedded evaluators can access or how they should report findings, and funding methods are unsettled. Over the longer run, it argued that funding should come from pooled or public sources.
Some want public oversight, not just voluntary deals.
Pieces of that model are showing up in legislation. IVOs are central to the "Frontier Risk Oversight, National Transparency, Independent Evaluation, and Reporting" Act, introduced in July by Reps. Lori Trahan, D-Mass., and Jay Obernolte, R-Calif. "It was important for them to hear that and hear it from us, and we wanted to be really clear about that," he said, according to reports.
States are moving, too.
For your wallet, here is the through-line: whoever sets the rules for independent evaluations will influence how fast risks are surfaced and how much trust buyers and regulators place in AI systems. If third-party oversight becomes credible and durable, that could shift where budgets go across the stack, from model developers to apps that need trustworthy assurances.
Who audits the models will shape which ones get deployed. Join Market Briefs free and follow it.
