News for hackers. Stuff that matters.

A comment on Ask HN: How are you extracting the best performance out of your RAG pipeline?. Parent comment

Re:Ask HN: How are you extracting the best performance out o... by imaravind on Sunday October 06, 2024 @09:26PM (#41761897)
@naveen99 That's awesome! But I'm curious how you went about data chunking, embedding etc.? Chose adhoc or applied some trial-and-error approach or something else?
[ Parent | on HN ]
Re:Ask HN: How are you extracting the best performance out o... by naveen99 on Monday October 07, 2024 @07:49PM (#41772352)
Yes generated embeddings using an LLM… no chunking. Tried a few of models. Clip and Imagebind didn’t work great for text, so went with a text focused one.
[ Parent | on HN ]
Re:Ask HN: How are you extracting the best performance out o... by chiccomagnus on Wednesday October 09, 2024 @08:56AM (#41787423)
By chance, have you tried preprocess.co for text extraction + chucking?
[ Parent | on HN ]