Re:Embeddings: What they are and why they matter
by SushiHippie on Wednesday October 25, 2023 @07:02AM
(#38011487)
I've found gpt4all to be quite good and runs locally.
Re:Embeddings: What they are and why they matter
by navbaker on Wednesday October 25, 2023 @08:27AM
(#38012051)
The HuggingFace embedding models are really good! See here for a good intro tutorial: https://huggingface.co/blog/getting-started-with-embeddings
Re:Embeddings: What they are and why they matter
by ahoho on Wednesday October 25, 2023 @08:27AM
(#38012053)
sentence-transformers is excellent. It will be slow on CPU but, because itβs on top of pytorch, does have apple silicon/GPU support.
Re:Embeddings: What they are and why they matter
by simonw on Wednesday October 25, 2023 @09:47AM
(#38012733)
I've been running these models on my own laptop without the GPU. They're generally a lot less resource demanding than full LLMs (though thanks to llama.cpp I've been able to run those on the CPU as well).
More about the tooling I use to run embedding models locally here: https://simonwillison.net/2023/Sep/4/llm-embeddings/
Re:Embeddings: What they are and why they matter
by kordlessagain on Wednesday October 25, 2023 @10:19AM
(#38013099)
Install Instructor and dependencies:
pip install InstructorEmbedding
pip install torch
pip install numpy
pip install tqdm
pip install sentence_transformers
Use it: from InstructorEmbedding import INSTRUCTOR
import numpy as np
from scipy.spatial.distance import cosine
xl = INSTRUCTOR('hkunlp/instructor-xl')
embeddings = xl.encode(["I'm amazed at how autoencoding actually works, despite the simplicity of the approach.", "Are there any good document embedding models I can run on my own hardware? "]).tolist()
cosine_distance = 1 - np.dot(embeddings[0], embeddings[1]) / (np.linalg.norm(embeddings[0]) * np.linalg.norm(embeddings[1]))
print(cosine_distance)
Output: 0.33516267980261427
Are there any good document embedding models I can run on my own hardware? Word2Vec is interesting, but I'd like to also try out cross-linking blog entries, etc.