Post

A new, highly recommended article from @rasbt.bsky.social: "Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attention" -> magazine.sebastianraschka.com/p/recent-dev... #MLsky

magazine.sebastianraschka.comRecent Developments in LLM Architectures: KV Sharing, mHC, and Compressed AttentionFrom Gemma 4 to DeepSeek V4, How New Open-Weight LLMs Are Reducing Long-Context Costs

Sebastian Raschka (rasbt)

Thanks!!

No problem, I am very grateful for your articles (and videos etc), I learn so much from them. BTW: A few days ago I stumbled across this "Simple Attention Networks" concept from Cactus, mostly for one-shot tool use, solely based on the provided context. -> github.com/cactus-compu...