simtj

OpenAI python package has a built in distances_from_embeddings function, that can be used to calculate cosine similarity or distance. javascript also has similar libs to do so.

From OpenAI Docs "Which distance function should I use? We recommend cosine similarity. The choice of distance function typically doesn’t matter much.

OpenAI embeddings are normalized to length 1, which means that:

Cosine similarity can be computed slightly faster using just a dot product Cosine similarity and Euclidean distance will result in the identical rankings"

Locally stored embeds does not get sent to the open ai call.

After having all the embeddings stored locally, You can begin asking it questions.

Upon asking a question, the question itself is sent to Open Ai for embedding, gets a response, compares against the embedding you stored locally, uses a function called cosine similarity to get the nearest distance of the embeddings (most relevant docs).

Then you link these sources together and send to openai again with the question now in text, to get a reply for your question based on the context.

there's no upper limit in theory. App cuts the pdf in chunks and batches ~ 8000 tokens on every call for embedding from open ai, and stores the vector values locally.

It takes quite abit of time to finish embedding 100k words, thats the most i've tried so far.

By using embeddings, relevant docs are clustered and then sent as context to query for a response