Finally, a Replacement for BERT

Benjamin Warner
1 min. read

https://huggingface.co/blog/modernbert

Previous
Enabling 70B Finetuning on Consumer GPUs
A Technical Deep Dive into FSDP+QLoRA

A detailed guide for adding FSDP and QLoRA support to quantization libraries and training frameworks.

Next
TIL: Masked Language Models Are Surprisingly Capable Zero-Shot Learners

I have a [MASK] and I must classify: using masked language modeling for downstream tasks works surprisingly well.