Post

dynomight
@dynomight.net

we're approaching superhuman coding and math but somehow LLMs are still mid at "writing"? this is undertheorized

0 reposts18 likes

nic 🔹
@nmk.wtf

A hypothesis: LLM writing gets saturated too quickly / doesn't sufficiently adapt to context? If you saw Hemingway text everywhere and in every context maybe we'd get sick of Hemingway too?

codehappy lives!
@codehappy.bsky.social

it is easy to directly do RL with verifiable reward in those domains; there is a correct answer and model responses can be scored automatically. it is also easy to synthesize new problems in those domains, and to estimate difficulty of each problem writing is different. reward function is messy

brennan
@brennan.computer

+1 over the years I've 'written' a lot of stories and tried to build tooling for recursively capturing style and such, and it's just so hard to carrot (or stick) behavior towards such a squishy definition of a goal

Ilya Farber
@nihilfit.bsky.social

Agree with others re: the reward function and synthetic examples; the other big one is that LLMs are already great at types of writing where the median / most-likely response is desirable, eg homework and bizslop. They only fall short in genres where we value novel ideas and style.

Ilya Farber
@nihilfit.bsky.social

That said, I personally am still floored by how good they are at writing to a given style and form. I think we got inured to this early since even the first public LLMs had it down. Granted there’s no RL use case for “write a sestina about good doggos in Poe’s language”, but it’s still astonishing.

makeworld
@makeworld.space

Good at formally specified, falsifiable scenarios, "bad" at subjective artistic ones? It doesn't sound so bizarre in that context, but it's all a strange world of course.