Post

Took some time today to catch up with Johann Rehberger's Month of AI Bugs and wow... 15 examples so far of major prompt injection vulnerabilities in products including ChatGPT, Codex, Cursor, Amp, Devin, Claude Code, GitHub Copilot and Google Jules simonwillison.net/2025/Aug/15/...

simonwillison.netThe Summer of Johann: prompt injections as far as the eye can seeIndependent AI researcher Johann Rehberger (previously) has had an absurdly busy August. Under the heading The Month of AI Bugs he has been publishing one report per day across an …

Martin Rauscher

@martin.rauscheronline.de

17 Aug 2025

I've wondered: why has no LLM been trained so far with special tokens that indicate start/end of content whose instructions should NOT be followed. It sounds like an obvious thing to do...

Simon Willison

@simonwillison.net

17 Aug 2025

They've tried but it doesn't work with 100% reliably

Martin Rauscher

@martin.rauscheronline.de

19 Aug 2025

Do you have a link or name to a paper? It might still be more reliable than prompting around it though

Simon Willison

@simonwillison.net

If it worked someone would definitely have told me about it! I've been writing extensively about this problem for nearly three years now The most recent attempt is the Harmony stuff OpenAI released for their gpt-oss models simonwillison.net/2025/Aug/5/g...

19/8/25, 7:13 pm0 reposts4 likes

Martin Rauscher

@martin.rauscheronline.de

20 Aug 2025

Haha, that's probably true, since you're pretty famous now 😅 Thanks, I'll have a read through it. Honestly that feels like very strong proof that alignment is an impossible issue...