Re:A Replacement for Bert
by bclavie on Friday December 20, 2024 @08:47PM
(#42476865)
Hey! It’s more like comparing apples to apple pie.
BGE-M3 is a fine-tuned embedding models. This means that they’ve taken a base language model, which was trained for just language modeling, then applied further fine-tuning to make it useful for a given application, in this case, retrieval.
ModernBERT is one step back earlier in the pipeline: it’s the language model that application-specific models such as M3 build on.
BGE-M3 is a fine-tuned embedding models. This means that they’ve taken a base language model, which was trained for just language modeling, then applied further fine-tuning to make it useful for a given application, in this case, retrieval.
ModernBERT is one step back earlier in the pipeline: it’s the language model that application-specific models such as M3 build on.