News for hackers. Stuff that matters.
Re:A Replacement for Bert by ZQ-Dev8 on Thursday December 19, 2024 @02:02PM (#42464634)
Jeremy, this is awesome! Personally excited for a new wave of sentence transformers built off ModernBERT. A poster below provided the link to a sample ST training script in the ModernBERT repo, so that's great.

Do you expect the ModernBERT STs to carry the same advantages over ModernBERT that BERT STs had over the original BERT? Or would you expect caveats based on ModernBERT's updated architecture and capabilities?

Re:A Replacement for Bert by jph00 on Thursday December 19, 2024 @02:04PM (#42464662)
Yes absolutely the same advantages -- in fact the maintainer of ST is on the paper team, and it's been a key goal from day one to make this work well.
Re:A Replacement for Bert by data_ders on Thursday December 19, 2024 @09:10PM (#42467633)
what’s ST stand for here? I googled and only got results for BERT STS (semantic text similarity)
Re:A Replacement for Bert by bclavie on Thursday December 19, 2024 @09:33PM (#42467749)
Sentence Transformers (https://sbert.net/), the most used library for embedding models (similarity, retrieval.)