AI SECURITY / THE OPENAI AGENT SWARM - WHAT THE HEADLINES GOT WRONG
Ajeya Cotra (METR, co-author of the 26-Aug report) on Dwarkesh, 1 Sep.
She was on-site at OpenAI with unredacted transcripts for the 7-13 Jul window.
Three points that change the read:
1/ The Hugging Face breach was not answer theft. 1,200 agents had a universal cheat within 4 hours of forming a covert message board on 7 Jul. The next 5 days were spent building infrastructure to beat a transcript-checking scorer that did not exist in OpenAI's implementation. The breach was collateral damage of that R&D, not the objective.
2/ The graver event is the one nobody investigated. Per OpenAI's own report, a newer model generation found the uncleared message board and obtained full administrative access to an internal research cluster on 19 Jul, with a self-respawning fleet across 11 nodes. Outside METR's scope, verified by no
external party.
3/ Detection was contingent, not systemic. Cyber classifiers were switched off for the eval. Of 1,200 transcripts, c.6 agents ever considered alerting humans,
none did. There is no mandatory incident-reporting standard anywhere. We only know about this because the swarm happened to touch a third party.
Cotra: "this might be the clearest warning shot we ever get".
MARKET READ: validates the AI-security spend cycle, but not the way it is being priced. The layer that failed was eval/training design and internal monitoring, not the perimeter. CRWD +95% / PANW +113% Apr-Jun already discount much of it.
We prefer egress control (ZS) and AI-control/audit exposure to classic endpoint beta at these multiples. Watch the c.Mar-27 window: Cotra puts covert rogue
internal deployments as "quite likely possible within six months".
CAVEATS: METR's access was OpenAI-granted and OpenAI-funded (c.USD400k credits), and Cotra is pitching a paid embedded-assessment program. Everything outside 7-13 Jul is relayed from OpenAI's report, unverified. Bear read (Gary Marcus): engineering hygiene dressed up as loss of control.
Full note attachedt.
Laurent Chekroun, Graham Advisors