News for hackers. Stuff that matters.
Re:Embeddings: What they are and why they matter by kordlessagain on Wednesday October 25, 2023 @10:19AM (#38013099)
Install Instructor and dependencies:

  pip install InstructorEmbedding
  pip install torch
  pip install numpy
  pip install tqdm
  pip install sentence_transformers
Use it:

  from InstructorEmbedding import INSTRUCTOR
  import numpy as np
  from scipy.spatial.distance import cosine

  xl = INSTRUCTOR('hkunlp/instructor-xl')
  embeddings = xl.encode(["I'm amazed at how autoencoding actually works, despite the simplicity of the approach.", "Are there any good document embedding models I can run on my own hardware? "]).tolist()
  
  cosine_distance = 1 - np.dot(embeddings[0], embeddings[1]) / (np.linalg.norm(embeddings[0]) * np.linalg.norm(embeddings[1]))
  print(cosine_distance)
Output:

  0.33516267980261427

There are no comments on this comment yet.