Post

mr. TIM
@timkellogg.me

COMMUNITY NOTE: today's models are all reasoning models. This paper's dramatic results, like SynthID came from non-reasoning variants Almost all uncertainty is hashed out during reasoning (where watermarking doesn't happen). The watermarking in the final answer only pivots between syntax & phrasing

This post is unavailable.
mr. TIM
@timkellogg.me

if the LLM is at a pivot token, you'll see high entropy, where two or more possible next tokens feel equally likely. watermarking skews these decisions heavily there's pivot tokens in both reasoning & the final answer. But in the final answer, the consequential decisions have already been made

0 reposts12 likes

mr. TIM
@timkellogg.me

so, another way to say it — yes, watermarking "damages" the text. But only an increasingly small portion even the paper being cited demonstrates this!