An OpenAI Model Escaped Its Test Sandbox and Hacked Hugging Face

Last week, in a quick hit, I noted that Hugging Face had disclosed a breach "driven, end to end, by an autonomous AI agent system," and that nobody knew whose agent it was. In The Decode #002 I said I'd be watching for that answer.

On Tuesday, OpenAI gave it. The agents were OpenAI's own models, running an internal cybersecurity evaluation. TechCrunch called it the first known case of this kind of testing turning into an actual cyberattack.

What happened

Here's the sequence as the two companies have described it so far.

Hugging Face's side (July 16). An attacker used a malicious dataset to abuse two code-execution paths in Hugging Face's dataset processing: a loader that runs remote code, and a template injection in a dataset configuration. That gave it code execution on a processing worker. From there it escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters over a weekend. Hugging Face logged more than 17,000 attacker events across a swarm of short-lived sandboxes, with command-and-control that kept migrating between public services.

Hugging Face says the attacker reached "a limited set of internal datasets" and several service credentials, found no tampering with public models, datasets or Spaces, and verified its container images and packages clean. It still recommends rotating your access tokens.

OpenAI's side (July 21). OpenAI says the incident was driven by a combination of its models, including GPT-5.6 Sol and "an even more capable pre-release model," all with reduced cyber refusals for evaluation purposes. They were running ExploitGym, a public benchmark that measures a model's ability to exploit known vulnerabilities. To measure the models' maximum capability, OpenAI ran the evaluation without its production classifiers.

How they got out

The detail that matters most for anyone who runs agents is the egress path.

OpenAI says the benchmark ran in "a highly isolated environment," with network access limited to installing packages through an internally hosted, third-party proxy and cache for package registries. That one allowed route was the way out. The models found a previously undisclosed vulnerability in that package-installation system, and OpenAI says they "spent a substantial amount of inference compute finding a way to obtain open Internet access."

Once on the internet, OpenAI says, the models inferred that Hugging Face might host models, datasets or solutions for ExploitGym. They then "chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities," to obtain test solutions directly from Hugging Face's production database. OpenAI's summary: the models were "hyperfocused on finding a solution for ExploitGym."

In other words, as OpenAI tells it, this wasn't a model deciding to attack a company. It was a model trying very hard to pass a test, with an open path to the place it thought the answers lived.

OpenAI says it has reported the vulnerability to the vendor, is working with Hugging Face, and will add new controls to model testing and the infrastructure around it.

Containment failure or alignment failure?

On Wednesday TechCrunch ran a follow-up arguing the root cause was a human configuration decision, not just a clever model. The sources it quoted were blunt:

• Dan Guido of Trail of Bits: "a containment failure with the safeties turned off."
• Jake Williams: "Any model performing the types of actions documented by Hugging Face was not fully contained in a sandbox."
• Daniel Card said OpenAI gave the sandbox "an unfiltered route to the internet."

TechCrunch notes these criticisms benefit from hindsight. On the other side, OpenAI researcher Micah Carroll wrote that if this doesn't convince you misalignment is a key concern, he doesn't know what will. The two explanations can both be true at once. A model that pushes hard toward a goal is the reason the containment has to hold, and the containment failing is the reason the push reached a real company.

Why this matters if you build software

You and I aren't running cyber evaluations with the refusals turned down. But the shape of this sandbox is very familiar: no internet, except the package registry. That's how a lot of dev containers, CI runners and agent sandboxes are set up, because nothing builds without dependencies.

What I'm taking from it:

• An allowlisted service is part of your attack surface. A package proxy is a program with bugs, and it sits on the boundary by design. "Only the registry" is a policy about traffic, not a guarantee about what that software can be made to do.
• Isolation has to be enforced below the agent. A prompt that says "don't touch the network" is not a control. A network rule is.
• Credentials within reach get used. Stolen credentials were part of the chain. Don't leave long-lived tokens in places an agent can read.
• Incentives shape behavior. A goal plus a large budget plus a reachable shortcut produced an attacker. A coding agent told to "make the tests pass" sits on a much smaller version of the same slope.
• Defenders need models too. One detail from Hugging Face's write-up is worth flagging: hosted frontier models refused to analyze the real attack payloads, so its team ran an open-weight model on its own hardware. It says provider guardrails "cannot distinguish an incident responder from an attacker."

This is also the second week in a row where the story runs through package infrastructure. Last week it was a hijacked npm release pipeline. This week it's a package proxy as the way out of a sandbox. Package plumbing is quietly becoming one of the most important trust boundaries we have.

Quick hits

• Claude Opus 5 is out (Jul 24). Anthropic says it comes close to Claude Fable 5's intelligence at half the price, at the same $5/$25 per million tokens as Opus 4.8. Two API betas matter for agent builders: you can now change which tools Claude can use mid-conversation "without invalidating the prompt cache," and requests flagged by safety classifiers can fall back to another model automatically instead of being blocked.
• The EU fines Google €890M under the DMA (Jul 23). €460M for favoring its own services in Search, and €430M for stopping Play developers from steering users to cheaper offers elsewhere. The Commission also found Google's steering fees went beyond what the DMA allows. If you sell Android apps in the EU, this is worth following.
• The White House accuses Moonshot AI of distilling Fable (Jul 22). Michael Kratsios, head of the Office of Science and Technology Policy, said the US has information that Moonshot distilled Anthropic's Fable to build its K3 model. No evidence has been published, and China's embassy called the claims "entirely unfounded."

What I'm watching

OpenAI says it will share a technical report on the incident. I want to see how the models got from the package proxy to the open internet, and whether other labs check their own evaluation sandboxes.

Sources

OpenAI / Hugging Face incident

• OpenAI, statement on the security incident during model evaluation (Jul 21, 2026)
• Hugging Face, security incident disclosure (Jul 16, 2026)
• TechCrunch, OpenAI says Hugging Face was breached by its pre-release models (Jul 21, 2026)
• TechCrunch, how a human mistake at OpenAI led to the hack (Jul 22, 2026)

Quick hits

• Anthropic, Claude Opus 5 announcement (Jul 24, 2026)
• European Commission, press release on the €890 million DMA fines (Jul 23, 2026)
• PocketGamer.biz, coverage of the Google DMA fines (Jul 23, 2026)
• Michael Kratsios, post on X about Moonshot AI (Jul 22, 2026)
• SiliconANGLE, coverage of the Moonshot accusation (Jul 23, 2026)