Post

Simon Willison
@simonwillison.net

Took some time today to catch up with Johann Rehberger's Month of AI Bugs and wow... 15 examples so far of major prompt injection vulnerabilities in products including ChatGPT, Codex, Cursor, Amp, Devin, Claude Code, GitHub Copilot and Google Jules simonwillison.net/2025/Aug/15/...

simonwillison.netThe Summer of Johann: prompt injections as far as the eye can seeIndependent AI researcher Johann Rehberger (previously) has had an absurdly busy August. Under the heading The Month of AI Bugs he has been publishing one report per day across an …
Jørgen Lund
@jaalu.bsky.social

Instructional Segment Embeddings arxiv.org/abs/2410.09102 kind of do this - they do not use separate start/end-of tokens, but add author embeddings on top of token embeddings during SFT - it does seem to make (small) models less vulnerable to malicious prompts, but it's an incremental improvement

arxiv.orgInstructional Segment Embedding: Improving LLM Safety with Instruction HierarchyLarge Language Models (LLMs) are susceptible to security and safety threats, such as prompt injection, prompt extraction, and harmful requests. One major cause of these vulnerabilities is the lack of ...