Presenting the trial as a way to scale collaborations with external safety organizations, METR and Redwood Research were allowed an atypically thorough window into OpenAI's internal systems. During the period from July 21 to Aug. 11, a single staff member from each nonprofit reviewed OpenAI's codebase and documentation and spoke with selected technical staffers, all under in-person supervision and on OpenAI-provided hardware that never left the facility.
The focus was OpenAI's internal misalignment monitoring system, which tracks whether AI behavior stays in line with human goals and values. The reviewers evaluated the version of the system in place up to July 21, and they observed that OpenAI was in the process of expanding it during the trial.
The teams reported several "novel vulnerabilities," including a way for agents to block monitoring of "hundreds" of recent actions. They provided a 61-page internal report to OpenAI and said the company told them it has fixed the vulnerabilities that affect its production safeguards.
Because the pilot was not intended to provide broad public transparency, METR and Redwood agreed to release only a high-level summary. They said they expect to publish more details about future exercises. "This collaboration builds on our longstanding commitment to pioneering new approaches to third-party testing, and we're using what we learned to strengthen our safeguards and shape future collaborations with independent evaluators," said an OpenAI spokesperson in a statement to CNBC.
When tech and policy overlap, staying informed matters, so sign up for the free Market Briefs newsletter for five-minute updates.
OpenAI CEO Sam Altman pledged in September to give independent evaluators "employee-like access," following Anthropic CEO Dario Amodei's proposal to slow AI development. Since then, Anthropic has announced agreements with Accenture, METR and SecureBio, and earlier this month OpenAI said it is "actively finalizing contracts with third-party safety assessors."
President Donald Trump, known for a light-touch stance on AI regulation, has likewise pushed for third-party reviews. He urged companies to "partner with an independent external auditor or evaluator" as part of a voluntary White House agreement that included signatures from Amodei and senior leaders at Google, Meta, OpenAI, SpaceX and Nvidia. As for the field itself, independent evaluators have traditionally operated in a quieter niche of the AI sector, and experts told CNBC that questions persist about how these groups are financed and how their reporting lines should be structured.
The bottom line: watch for formal contracts and repeatable testing programs. Moves that normalize independent evaluations could influence disclosure norms and perceived risk across the AI stack, which can ripple into valuations, procurement decisions and who wins enterprise spend.
Even when details are complex, a quick daily briefing can help, subscribe to Market Briefs for concise market and tech context.
