Post

What do LLMs see? I wrote a lil' tool that extracts the attention matrices out of open models and creates this typing visual, with each token's opacity changing according to its average attention score as the prompt progresses. Dimmer words are considered less important to the model.

Andreas Møller

@andreasmoller.dk

19 Feb

So they are narcissists.

Tim Duffy

@timfduffy.com

19 Feb

LLMs commonly use the first token as an attention "sink", giving it a high attention score regardless of what token it is. arxiv.org/abs/2504.02732

arxiv.orgWhy do LLMs attend to the first token?Large Language Models (LLMs) tend to attend heavily to the first token in the sequence -- creating a so-called attention sink. Many works have studied this phenomenon in detail, proposing various ways...

🌍 Duncan Kimpton 🛠️

@dgkimpton.bsky.social

19 Feb

Does that imply that beginning a prompt with the most significant topic word might help get more focused results? Or am I missing the point?

Tim Duffy

@timfduffy.com

My guess is no. The effect of the sink on the first token isn't so much about the first token itself but about moderating attention interactions in other token positions.

19/2/26, 9:10 pm0 reposts4 likes

🌍 Duncan Kimpton 🛠️

@dgkimpton.bsky.social

19 Feb

Oh, interesting. There's so much I don't know about attention in llms it boggles my mind and seems to defy intuitive understanding.