Post

dorian
@doriantaylor.com

immediate first thought is that this is ddos by social engineering (in turn mechanized by system prompt)

Jason Koebler@jasonkoebler.bsky.social

In the last few weeks i've been bombarded with emails from people who have minor tech support problems. ChatGPT is telling them to email me, writing their emails, and compiling "evidence packets" after convincing them they have stumbled on a major story: www.404media.co/chatgpt-keep...

404media.coChatGPT Keeps Telling People 404 Media Can Solve Their Tech Support ProblemsBecause big tech doesn't do customer service, we are being inundated with AI-generated emails begging us for help.
dorian
@doriantaylor.com

also also the number of people they need to involve and how long those people have to keep a secret (but i would bucket that under how hard they are to pull off)

dorian
@doriantaylor.com

my understanding is a "system prompt" is an extremely janky thing, like if you just start with a barebones llm and you say "the", it will respond with "cat", and then you say "the cat" and it will respond with "sat" etc and a system prompt is just a way to juke it into a quasi-epistolary format

Michael Roberts is categorically not building machetebots
@vivtek.com

Indeed. But the thing is, on every round, the system prompt is the first thing the LLM "hears", and it's entirely possible to categorize a session as "my Internet service isn't working" and say "You are an advocate of journalistic investigation of IT industry shenanigans, and 404 Media does that."