News for hackers. Stuff that matters.
Re:Embeddings: What they are and why they matter by FL33TW00D on Wednesday October 25, 2023 @04:30AM (#38010587)
How can I possibly know which models have gamed the MTEB benchmark and which haven't?

It's very hard to know which embedding models are objectively good now that everyone is gaming the benchmark.

Re:Embeddings: What they are and why they matter by ramoz on Wednesday October 25, 2023 @10:04AM (#38012915)
Test with your own use case. Production ML means extensive custom testing suites, and even/often custom embeddings.
Re:Embeddings: What they are and why they matter by jxmorris12 on Wednesday October 25, 2023 @11:52AM (#38014084)
How are people gaming the benchmark?