Post

Having an interesting chat with Claude, I asked about accuracy of LLMs and it answered (in the first person) some fun admissions: "Treat with scepticism anything where I sound confident but you can't easily check — confidence in my outputs is not well calibrated to accuracy" 😯 #UKEd #TeamCompSci

0 reposts0 likes

It goes on (ignore the bit about a book I'm probably not going to write...) "Even where I'm reliably accurate, I'm producing text that pattern-matches to correctness without the kind of understanding that would let me know why it's correct or when to doubt it. " More in Alt Text #TeamCompSci #UKed

There was an article I read somewhere (maybe the Guardian magazine) about how the biggest change in the last year is that LLMs no longer say they don't know - they'll confidently give you an answer that they're not really sure about.

I was out with some Computing people last week. Two of them are no longer teachers - one does something with AI. He said he uses AI all the time and can spend £200-300 per day on tokens. When I said that I never use it, he said "I probably wouldn't use it if I had a proper job, like teaching."