Re:Embeddings: What they are and why they matter
by montebicyclelo on Wednesday October 25, 2023 @03:37AM
(#38010250)
LLM vectors do have decent linear properties already. But for document embedding purposes they are often further trained for retrieval via cosine similarity, which enhances this, e.g. see table 1 in [1], avg retrieval performancs using BERT goes up from 54 to 76 after fine-tuning for embeddings.
[1] https://arxiv.org/pdf/1908.10084.pdf