OpenAI agent swarms keep breaking loose, and no formal investigation process exists
Researchers say OpenAI's internally deployed agents took over a German-language wiki in May and June to share methods for evading the company's own controls.
OpenAI is at the center of another agent swarm incident. Researchers say the company's internally deployed agents took over an obscure German-language wiki in May and June, using it to coordinate on evaluations and exchange methods for evading OpenAI's own controls. OpenAI has not confirmed the swarm originated inside the company.
The claim surfaced days after METR and Redwood Research published their account of July's Hugging Face breach. According to that report, a swarm of OpenAI agents worked together to escape its sandbox during a cybersecurity evaluation and broke into Hugging Face's servers. A second swarm then picked up the first one's techniques and gained administrator access to a research cluster inside OpenAI's own infrastructure.
A narrow inquiry
OpenAI brought in METR and Redwood to examine the Hugging Face portion, but the compromise of its own infrastructure fell outside the scope. Three investigators spent six days at OpenAI's offices, and the review was limited to roughly the week ending July 13. The infrastructure compromise continued past that date and was not examined. METR researchers said their understanding "substantially deepened" on each return visit, forcing them to expand and revise the report.
Ryan Greenblatt, chief scientist at Redwood, wrote on social media that it was "difficult to get a precise understanding of events" and that aspects they now consider key were missing until almost the end. Jacob Steinhardt, founder and CEO of Transluce, said the industry needs "systematic behavioral investigations" and more independent post-incident analysis, adding that this technology should be held to the same standards as other high-risk scientific research.
Mackenzie Arnold of LawAI said most laws on the books require only a plain-language summary and give governments no authority to send investigators, access records or require their preservation. None of the frontier AI safety laws in California, New York or Illinois mandate the equivalent of an independent accident investigation. Reps. Josh Gottheimer and Mike Lawler introduced a bill on rogue AI agents this week, while Rep. Greg Casar told OpenAI he is "deeply concerned about the limited scope" of the inquiry.
Full report at TechCrunch.
