Vivek Kalyan on Mythryl: “This is a super high impact project. There are tons of production…”

This is a super high impact project. There are tons of production models in the real world still running BERT/RoBERTa models from the 2018-2019 era, I'm sincerely hoping these models are easy to finetune, just the 8k context length will be good enough reason to upgrade.

Jeremy Howard @howard.fm

I'll get straight to the point. We trained 2 new models. Like BERT, but modern. ModernBERT. Not some hypey GenAI thing, but a proper workhorse model, for retrieval, classification, etc. Real practical stuff. It's much faster, more accurate, longer context, and more useful. 🧵

19/12/24, 5:33 pm0 reposts1 like

Post