Post

ModernBERT is officially released on Transformers v4.48.0. You no longer need to install from git to use. If you are plugging ModernBERT into an existing encoder finetuning pipeline, try increasing the learning rate. We've found that ModernBERT tends to prefer a higher LR than older models.

10/1/25, 6:28 pm3 reposts11 likes

Benjamin Warner

@benjaminwarner.dev

10 Jan 2025

What's ModernBERT? It's a drop-in replacement for existing BERT models, but smarter, faster, and supports longer context. Check out our announcement post for more details: huggingface.co/blog/modernb...

huggingface.coFinally, a Replacement for BERT: Introducing ModernBERTWe’re on a journey to advance and democratize artificial intelligence through open source and open science.