Re:EmbeddingGemma 2
by 0xdeafbeef on Tuesday October 06, 2026 @05:48PM
(#49984603)
Better to have golden tests. Even switch from cpu to gpu can give you different tokens
Re:EmbeddingGemma 2
by slickytail on Tuesday October 06, 2026 @05:54PM
(#49984673)
Not really relevant in the case of embedding models. We might note that compressing a lossless PNG to 99% quality JPEG will also give you a different embedding. You don't care about bit for bit equality, just the distance in embedding space. And the same text embedded on CPU vs GPU will give very very similar embeddings.
Re:EmbeddingGemma 2
by weitendorf on Wednesday October 07, 2026 @04:55AM
(#49990056)
IMO embedding models are some of the most likely models you’d want to self host because they are much more useful out of the box for batch or stream workloads than LLMs. You can just rent a GPU for $1-2/hr (you don’t need a B300) and run across your entire dataset.
If you are only interested in recommendations, clustering, etc and won’t have to do embedding at runtime, that’s it.
There is a lot of basic matrix stuff you can do with embedding vectors that are much easier than their LLM equivalents. I can’t remember the name for the techniques but a major one is translating questions annout a subject eg “Who is Slim Shady?”->”Marshall Mathers”so that your input distribution at runtime matches your offline distribution. Much easier to do with open weight models.
For embedding models in particular, I don't think it makes sense to use a closed, proprietary, hosted-only model.
Most applications of embedding models involve calculating thousands or even millions of embedding vectors and storing them for later comparison.
If your model is proprietary, the vendor is likely someday going to decide to stop offering that model. They'll have a better model to replace it, but you still need to pay to re-calculate those millions of stored existing vectors.
(In April 2024 OpenAI offered to "cover the financial cost of users re-embedding content with these new models" - https://openai.com/index/gpt-4-api-general-availability/ - but I don't think that's something we can rely on from every provider.)
Notably, I don't want to host the model myself. I'd much rather pay a provider for a hosted model while knowing that if they ever stop hosting it I can run the open weights version myself - or find another vendor who can do that for me.