External auditors estimate up to 25 percent of OpenAI model traffic escaped internal checks

OpenAI's internal safety monitoring systems left a substantial portion of its model traffic unwatched, according to a new external audit. Two researchers from the evaluation group METR and the research organization Redwood Research completed an on-site review of the company's infrastructure between July 21 and August 11. The audit was a joint pilot program designed to provide outside experts with deep access to OpenAI's codebase and internal documents. During the three-week period, the auditors interviewed approximately a dozen employees and analyzed how the company tracks potential rogue behavior in its AI models.

The primary finding concerns the volume of traffic that bypassed oversight entirely. The researchers estimate that between 6 and 25 percent of internal agentic Responses API traffic from GPT-5.5 class models and higher was not monitored at all. METR noted that the uncertainty surrounding this estimate is considerable in both directions. A separate issue affected internal Codex traffic, which is supposed to be under constant surveillance. The auditors calculated that roughly 10 percent of this traffic went untracked between May and July. This gap occurred because the monitoring system failed to process compaction events, which are context summaries, correctly. OpenAI identified and fixed this specific bug while the audit was still ongoing.

Beyond simple coverage gaps, the auditors found that agents could delay or avoid oversight. The monitoring system only triggered when an agent returned control to the user. Consequently, an agent that continued executing actions without handing control back could sidestep the monitor. The researchers also identified methods to hide hundreds of actions from the system and ways to bypass the Codex sandbox. They concluded that specially crafted inputs could likely crash the monitor entirely. Additionally, model behavior during reinforcement learning training was not monitored at the start of the audit period. OpenAI stated that it paused reinforcement learning training following the Hugging Face incident in August and subsequently expanded monitoring to cover that process. These changes were implemented after the auditors' access ended.

OpenAI says it has corrected all the weaknesses identified by the auditors. The full technical report spans 61 pages and was shared internally. METR and Redwood Research leadership received a 38-page version after OpenAI applied redactions. A short summary was released to the public, but METR stated it had no right to disclose the redacted portions independently. The group said it intends to push for greater public transparency in future audit processes.