Post

Pretty much every criticism of AI also sounds like a criticism of 90% of human behaviour

I think it’s reasonable to expect new technology to be better than us, I can understand being frustrated when you ask it something and it comes back with total nonsense. I don’t think it’s a great defence to say “people also make mistakes.”

But it IS better than us, that’s why we are both using it so much. It’s just not better than us at *everything*

For example, it’s very common now for frontier models to be good at knowing when they need to go and retrieve more information. Here’s a very simple example. For higher stakes questions, you definitely want to be clicking through to the sources to make sure you’re getting good information.

But isn't that orders of magnitude more processing power and electricity use than going direct to the Met Office website yourself to get the weather forecast? With a chance it could still get it wrong. That seems like the inferior option, or is this take outdated now too?

If you relied on the model weights directly for fact retrieval then yes, you’re going to get garbage. The mental model I’ve settled on for this is that information contained in the model itself is exceptionally fuzzy and compressed. Information in the context window, though, is very clear.

I got this mental model from listening to Andrej Karpathy on the Dwarkesh podcast. It’s a good listen if you’re interested in this topic. youtu.be/lXUZvyajciY?...