Post

I wrote about the completely wild incident where OpenAI were testing a new model and it broke out of its sandbox and broke INTO Hugging Face to steal the answers to the benchmark simonwillison.net/2026/Jul/22/...

Tucked away in this article is an appeal to the AI skeptics to PLEASE stop writing off stories like this OpenAI accidental exploit of Hugging Face as a dishonest marketing trick Frontier models can find and exploit vulnerabilities now, it helps nobody to pretend that they can't!

If I had built a system capable of autonomously breaking out of a sandbox and hacking another company’s systems, you can bet your arse I’d use it as marketing. It’s the most awesome marketing opportunity in the world. Scary, but awesome. I’ve no doubt it happened, so it’s not dishonest.

If I had open source competitors now catching up of over taking me just before I IPO I’d be bullshitting about having something special too. In every direction, Anthropic and OpenAI are being caught with their pants down because they’ve hit an inflection point of scale, resources, and cash flow

Yeah I also suspect the actual goal is to hinder or criminalize the distribution of open weights models. They are the biggest risk for their business

But in OpenAI and HF's discussion, they explicitly pointed out that HF detected and countered the breach using open weight models - because the closed models refused to engage due to security guardrails. So...that argument doesn't make a lot of sense.

It makes total sense. If they drum up enough hype about security risk of frontier models, the orange turnip and his mates will do something silly like ban their use entirely in the US unless under contract by the GOV. OpenAI and Anthropic already have contracts so they’ll be fine