News for hackers. Stuff that matters.
Re:Embeddings: What they are and why they matter by mikewarot on Wednesday October 25, 2023 @06:50AM (#38011411)
I'm amazed at how autoencoding actually works, despite the simplicity of the approach. I've dabbled with this stuff quite a bit, but I tend towards stuff I can run on my own computer, rather than trusting any service to remain stable for years or decades.

Are there any good document embedding models I can run on my own hardware? Word2Vec is interesting, but I'd like to also try out cross-linking blog entries, etc.

Re:Embeddings: What they are and why they matter by SushiHippie on Wednesday October 25, 2023 @07:02AM (#38011487)
I've found gpt4all to be quite good and runs locally.

https://docs.gpt4all.io/gpt4all_python_embedding.html

Re:Embeddings: What they are and why they matter by navbaker on Wednesday October 25, 2023 @08:27AM (#38012051)
The HuggingFace embedding models are really good! See here for a good intro tutorial: https://huggingface.co/blog/getting-started-with-embeddings
Re:Embeddings: What they are and why they matter by ahoho on Wednesday October 25, 2023 @08:27AM (#38012053)
sentence-transformers is excellent. It will be slow on CPU but, because it’s on top of pytorch, does have apple silicon/GPU support.
Re:Embeddings: What they are and why they matter by simonw on Wednesday October 25, 2023 @09:47AM (#38012733)
I've been running these models on my own laptop without the GPU. They're generally a lot less resource demanding than full LLMs (though thanks to llama.cpp I've been able to run those on the CPU as well).

More about the tooling I use to run embedding models locally here: https://simonwillison.net/2023/Sep/4/llm-embeddings/

Re:Embeddings: What they are and why they matter by kordlessagain on Wednesday October 25, 2023 @10:19AM (#38013099)
Install Instructor and dependencies:

  pip install InstructorEmbedding
  pip install torch
  pip install numpy
  pip install tqdm
  pip install sentence_transformers
Use it:

  from InstructorEmbedding import INSTRUCTOR
  import numpy as np
  from scipy.spatial.distance import cosine

  xl = INSTRUCTOR('hkunlp/instructor-xl')
  embeddings = xl.encode(["I'm amazed at how autoencoding actually works, despite the simplicity of the approach.", "Are there any good document embedding models I can run on my own hardware? "]).tolist()
  
  cosine_distance = 1 - np.dot(embeddings[0], embeddings[1]) / (np.linalg.norm(embeddings[0]) * np.linalg.norm(embeddings[1]))
  print(cosine_distance)
Output:

  0.33516267980261427