News for hackers. Stuff that matters.
Re:GPT-3.5 Turbo fine-tuning and API updates by stavros on Tuesday August 22, 2023 @04:27PM (#37228107)
Speaking of RAG, does anyone know of a Python library that (hopefully) doesn't need a server, that can do embeddings, along with an example? I want to do RAG in Python, and I can wrangle the prompt to where I want it, but I have no idea how to basically search for relevant documents with embeddings.
Re:GPT-3.5 Turbo fine-tuning and API updates by lukev on Tuesday August 22, 2023 @04:39PM (#37228208)
https://www.trychroma.com

It works as a library, you can require the PIP package and get started immediately.

Re:GPT-3.5 Turbo fine-tuning and API updates by stavros on Tuesday August 22, 2023 @04:42PM (#37228237)
This looks great, thank you!
Re:GPT-3.5 Turbo fine-tuning and API updates by J_Shelby_J on Tuesday August 22, 2023 @08:25PM (#37230375)
Here is my implementation used for my discord and slack bots. The logic for rag is the Shelby_agent module.

RAG is straightforward but i improve it by generating extra keywords to help the search. I also add extra metadata filtering to return both technical documentation and more fluffy sales and marketing stuff. IMO, semantic search alone might not be the best, and I want to also add keyword search.

I think the thing most people will struggle with is prepping the documents. Chunking them to the best token count for your use case for example.

https://github.com/shelby-as-a/shelby-as-a-service

Re:GPT-3.5 Turbo fine-tuning and API updates by stavros on Tuesday August 22, 2023 @08:27PM (#37230393)
That's very helpful, thanks!
Re:GPT-3.5 Turbo fine-tuning and API updates by stavros on Tuesday August 22, 2023 @09:58PM (#37230996)
I know I left another comment, but I'm replying in a new one to increase the chance that you'll see this: It would probably be very useful to a lot of people if you (or someone) made a standalone library out of this. It looks like Shelby requires Pinecone, but using something like ChromaDB for local embeddings and providing a library that I can just import, add documents to, and then query for answers would enable all kinds of applications.

I'm currently trying to build a hobby virtual assistant, and it looks like RAG does need some nuance to implement correctly (from what you said). A library that does all of what you mentioned would take a lot of work out of it.

Re:GPT-3.5 Turbo fine-tuning and API updates by J_Shelby_J on Wednesday August 23, 2023 @01:32PM (#37239400)
That's actually something that I think is a next step.

a) run locally through a web ui b) run with a local vectorstore

From my perspective of someone using it as a bot that runs in a container using a cloud based vector store makes a lot of sense, but yeah that's really good feedback and not something I considered. Hrmmm. Ok let me get a look at some local vectorstores.