Post

The progress of AI most importantly.

what was your view of AI progress 10 years ago?

I didn't have the vision that saturating any objective we put in front of it would naturally result in it doing things way beyond the tasks we were already deploying ML to do. I sort of thought "Existing stuff gets better. We're still mostly doing binary classification." bsky.app/profile/moul...

My first WTF moment along those lines was T5, where I started seeing how these things could be truly general. en.wikipedia.org/wiki/T5_(lan...

But then even after gpt3 had what to me seemed like obviously actual intelligence, I didn't do the extrapolation and really prepare in any way. I think I was scared of thinking about it.

for me it's only really been the last month or two using Opus 4.5 that I started to realise even the most scifi-esque of predictions about AI were going to come true after all

say more? :)

(why did using a specific model 'suddenly' do it, rather than general trend over improvement in models even far over the horizon)

this might make me look ir-Rational, but I've found it a useful heuristic to not extrapolate every line infinitely far into the future. it was less an opinionated stance that progress would halt once AI started approaching human-levels bc of limitations in the training corpus, and more just, like...

if you're using LLMs to write code or explain things and it always falls short of your own abilities at some point.. it doesn't really matter whether it failed way before your own capabilities or really close to them. the heuristic "this isn't as clever as me" solidifies. it's hard to look past that

progress is linear yes, but AI being smarter than me was always false false false, and it will only become true once (and then remain true forever), so when it does happen that's a black swan event almost by definition