Ask HN: How are you extracting the best performance out of your RAG pipeline?

Re:Ask HN: How are you extracting the best performance out o... by naveen99 on Thursday October 03, 2024 @10:46PM (#41737301)

We got good enough performance for semantic search just with faiss.

example similar comments to your submission text: https://hn.garglet.com/similar/comment/41727287

example random query: https://hn.garglet.com/form/textSearch?input6=We+got+good+en...

example, similar users to me: https://hn.garglet.com/similar/users/naveen99

[ Parent | on HN ]

Re:Ask HN: How are you extracting the best performance out o... by imaravind on Sunday October 06, 2024 @09:26PM (#41761897)

@naveen99 That's awesome! But I'm curious how you went about data chunking, embedding etc.? Chose adhoc or applied some trial-and-error approach or something else?

[ Parent | on HN ]

Re:Ask HN: How are you extracting the best performance out o... by naveen99 on Monday October 07, 2024 @07:49PM (#41772352)

Yes generated embeddings using an LLM… no chunking. Tried a few of models. Clip and Imagebind didn’t work great for text, so went with a text focused one.

[ Parent | on HN ]

Re:Ask HN: How are you extracting the best performance out o... by chiccomagnus on Wednesday October 09, 2024 @08:56AM (#41787423)

By chance, have you tried preprocess.co for text extraction + chucking?

[ Parent | on HN ]