Ask HN: How to manage docs for LLM RAG app?
These can be ever-evolving and may hook up with periodic delta updates, manual sync, add/remove, etc. And I'm trying to figure out if there's a way to manage these docs/texts properly. Basically, I think I would need a system to store these files, their metadata, etc, and provide a web UI for people to manage them. Then these blob of texts will go through frameworks like langchain/LlamaIndex and be cleaned/chunked into vector db, and different chunking strategies can be A/B tested while other people maintain this ever-growing docs system.
Any suggestions are welcomed. I've tried some all-in-one frameworks but so far my experience are lackluster. Also, my company due to compliance constraints cannot use cloud-based solutions, so it has to be either open-source local-deployed, or developed locally.
https://dify.ai/
And I think this results in my observation of the startup llm app world vs the heavily-regulated finance industry world: that the startup llm world is taking a default opt-in, default saas mode; however the regulated clients cannot benefit from these practices due to arcane regulation requirements. In this case, dify requires the setup of an API key for a service that I don't intend to use, and I have to go through the hurdles just to get the hello world going.
My analysis is that, there are 2 ways to make the heavily-regulated clients happy: 1. major cloud provider integrating everything into the myriad of its service catalogs and offer a full suite of services with the sales and compliance completed as a whole. 2. startup go down a curated, opinionated, battery-included oss approach with the least of sass fuss.
2. for some reason when I test using the simplest QA template, it seems dify cannot retrieve the document of any kind that I uploaded. I don't know why it happened but debugging the simplest offering of an app that boasts ease of use is not my thing.
I didn't go further from there. I wish my experience is just an extremely rare case since dify definitely seems to be the most polished all-in-one provider out there. If my experience turned out to be more positive I'll definitely stick with it, now I just feel picking a all-in-one is not the right choice.