Post

New post! ✨ Prompt caching ✨ My first big project post for @ngrok.com. 5 weeks, 12217 lines of code, 195 commits. I poured a lot into this one, and learned a lot in the process. I really hope you enjoy it ❤️

Oh that is so good! Congrats on getting it out

Thank you 🙏❤️

The attention part was new to me. I’m trying to understand why and how they “project” now: how do they write a poem where the last word rhymes? They do and we know that, and my simplistic understanding is they somehow project forward. But how does that rhyme 🙌 with token by token?

Yea that stuff is just fascinating…