Re:Ollama: Embedding Models
by barrell on Tuesday April 09, 2024 @09:21AM
(#39979169)
Even in this example, I don’t think that the middle of lions, tigers, and bears would be further away from lions than tigers.
But it might also help to understand that most often your embedding smaller chunks of data, specifically for the reason of specificity. A lot of embedding models have a hard limit of 512 tokens or so.
Embedding text that is incredibly short, or much longer than that, tends to decrease the quality precisely because of what you’re talking about. Although I don’t think to the extent you’re imagining
But it might also help to understand that most often your embedding smaller chunks of data, specifically for the reason of specificity. A lot of embedding models have a hard limit of 512 tokens or so.
Embedding text that is incredibly short, or much longer than that, tends to decrease the quality precisely because of what you’re talking about. Although I don’t think to the extent you’re imagining