Post

Speculative Decoding: The Art of Guessing Well Speculative decoding lets smaller models guess several tokens ahead while the larger model verifies them in parallel, reducing LLM inference latency without changing the… https://astralord.github.io/posts/speculative-decoding-the-art-of-guessing-well/