Show HN: Chat with multiple (100k words) PDFs privately

Re:Show HN: Chat with multiple (100k words) PDFs privately by andrewfromx on Tuesday April 11, 2023 @01:22PM (#35528201)

and this code that "compares against the embedding you stored locally, uses a function called cosine similarity" is open source from openai or something you wrote and not open source?

[ Parent | on HN ]

Re:Show HN: Chat with multiple (100k words) PDFs privately by simtj on Tuesday April 11, 2023 @01:41PM (#35528473)

OpenAI python package has a built in distances_from_embeddings function, that can be used to calculate cosine similarity or distance. javascript also has similar libs to do so.

From OpenAI Docs "Which distance function should I use? We recommend cosine similarity. The choice of distance function typically doesn’t matter much.

OpenAI embeddings are normalized to length 1, which means that:

Cosine similarity can be computed slightly faster using just a dot product Cosine similarity and Euclidean distance will result in the identical rankings"

[ Parent | on HN ]

Re:Show HN: Chat with multiple (100k words) PDFs privately by andrewfromx on Tuesday April 11, 2023 @01:49PM (#35528587)

thank you very much for all this info!

[ Parent | on HN ]