News for hackers. Stuff that matters.
Re:Dot: use of local LLMs and RAG in particular to interact ... by turnsout on Sunday April 07, 2024 @08:48AM (#39960400)
Curious about the choice of FAISS. It's a bit older now, and there are many more options for creating and selecting embeddings. Does FAISS still offer some advantages?
Re:Dot: use of local LLMs and RAG in particular to interact ... by alexpinel on Sunday April 07, 2024 @12:01PM (#39961612)
Hi! I'm the guy who made Dot. I remember experimenting with a few different vector stores in the early stages of the project but decided to settle with FAISS. I mainly chose it because it made it easy to perform the whole embedding process locally and also because it allows to merge vector stores which is what I use to load multiple types of documents at once. But I am definately not an expert on the topic and would really appreciate suggestions on other alternatives that might work better! :)
Re:Dot: use of local LLMs and RAG in particular to interact ... by simonw on Sunday April 07, 2024 @10:52AM (#39961163)
What options do you think work better?
Re:Dot: use of local LLMs and RAG in particular to interact ... by turnsout on Sunday April 07, 2024 @11:45AM (#39961516)
I don’t have an opinion, just wondering why they didn’t choose other another option such as Sentence Embeddings, OpenAI embeddings, etc.
Re:Dot: use of local LLMs and RAG in particular to interact ... by simonw on Sunday April 07, 2024 @06:34PM (#39964556)
FAISS can be used with OpenAI embeddings (and any other embedding model).

FAISS is technology for fast indexed similarity vector search - using it is an independent decision from which model you use to create those vectors.

Re:Dot: use of local LLMs and RAG in particular to interact ... by J_Shelby_J on Sunday April 07, 2024 @11:02AM (#39961225)
I’m trying build an exhaustive list of realistic options. See the spreadsheet here

https://shelbyjenkins.github.io/blog/retrieval-is-all-you-ne...