Post

Something I found interesting about reading this thread is that it made me wonder how many people think of ChatGPT as primarily a tool for writing things for you I use ChatGPT/Claude/Gemini possibly 100+ times a day and NONE of those uses are to produce writing that I then share with other people

... It's a text generator? Like that's its primary purpose, generating plausibly-human text from a prompt. It's both designed to do that and advertised for that. If it's not primarily a text generator then what is it for you?

I think "generative AI" is a bit of a misleading name - I think of it more as "transformative AI", the most interesting use-cases are when you pipe a bunch of text INTO the model and then instruct it to transform or process that text in some way

The simplest example is answering questions against documents - e.g. when I used it to help me find specific parts of the EU AI act relating to the term "open source" simonwillison.net/2025/Apr/19/...

Feeding in larger chunks of code and using it to help analyze potential sources of bugs is another way to use it like that - I did that yesterday with some test failures and Gemini 2.5 Flash gist.github.com/simonw/d867a...

To me this is always equal parts amazing and terrifying because it reminds me of this talk (the German one at least): www.dkriesel.com/en/blog/2013... “Never trust a scan you haven’t faked yourself” gets a lot more scary when a scan is not as easily recognizable as such.

I think it's essential to keep the e originals around because if you're going to be reporting on those numbers (as a jour almost) or making decisions based on them you need to be able to spot-check - just like if you had used human data entry teams

Generally though we've been wanting to build software that "understands" human language for decades - LLMs do a good enough imitation of that to be genuinely useful, for plugging together all sorts of tools and processes where a human can tell the computer what they want to do

I like this rule of thumb from @seldo.com seldo.com/posts/what-i...

I think the one counterexample is that it's good at writing a lot of code, which is creating more text. My rough rule of thumb is that LLMs are good when it's easy to verify the output as correct, which is quite a lot of things

I’m not convinced they are good at it. Or at least the ones I’m using can’t reliably preserve semantics. I keep getting summaries that drop negatives and invert the meaning, or treat speculation as fact and misstate conclusions.