News for hackers. Stuff that matters.
Re:Introducing GPTs by dragonwriter on Monday November 06, 2023 @03:21PM (#38168404)
If you are using OpenAI, the new Assistants API looks like itnwill handle internally what you used to handle externally with a vector DB for RAG (and for some things, GPT-4-Turbo’s 128k context window will make it unnecessary entirely.) There are some other uses for Vector DBs than RAG for LLMs, and there are reasons people might use non-OpenAI LLMs with RAG, so there is still a role for VectorDBs, but it shrunk a lot with this.
Re:Introducing GPTs by oezi on Monday November 06, 2023 @04:36PM (#38169369)
OpenAI is still way too expensive to run a corporate knowledge base on top
Re:Introducing GPTs by m3kw9 on Monday November 06, 2023 @09:19PM (#38172362)
It’s more reliable than chatpdfs that relies on vector search. With vector db all you are doing is doing a fuzzy search and then sending in that relevant portion near that text and send it to a LLM model as part of a prompt. It misses info.
Re:Introducing GPTs by dragonwriter on Monday November 06, 2023 @11:57PM (#38173415)
I'd be very surprised if the Assistants API is not doing RAG with a vector DB behind the scenes with the supplied files.