News for hackers. Stuff that matters.
Re:Ask HN: How to manage docs for LLM RAG app? by omidh on Thursday August 08, 2024 @02:04AM (#41188564)
Did you try dify? I found it was a good beginning for me.

https://dify.ai/

Re:Ask HN: How to manage docs for LLM RAG app? by sorcerer69 on Thursday August 08, 2024 @10:30AM (#41191951)
you own it dude? Damn crazy good UI
Re:Ask HN: How to manage docs for LLM RAG app? by Jianghong94 on Thursday August 08, 2024 @10:32PM (#41198361)
Yes. There're a couple of problems I encountered during my trial and ended up choose to build my own: 1. the version I was using requires the configuration of a 'reranking model', which AFAIK has to be from cohere. Funny enough, I can later opt to not use this reranking model, but I cannot get any functionality working if I don't preconfigure the cohere API key for this model. The problem with this is that, I need to request my company's infosec to allow API call onto cohere, for a functionality that I didn't intend to use.

And I think this results in my observation of the startup llm app world vs the heavily-regulated finance industry world: that the startup llm world is taking a default opt-in, default saas mode; however the regulated clients cannot benefit from these practices due to arcane regulation requirements. In this case, dify requires the setup of an API key for a service that I don't intend to use, and I have to go through the hurdles just to get the hello world going.

My analysis is that, there are 2 ways to make the heavily-regulated clients happy: 1. major cloud provider integrating everything into the myriad of its service catalogs and offer a full suite of services with the sales and compliance completed as a whole. 2. startup go down a curated, opinionated, battery-included oss approach with the least of sass fuss.

2. for some reason when I test using the simplest QA template, it seems dify cannot retrieve the document of any kind that I uploaded. I don't know why it happened but debugging the simplest offering of an app that boasts ease of use is not my thing.

I didn't go further from there. I wish my experience is just an extremely rare case since dify definitely seems to be the most polished all-in-one provider out there. If my experience turned out to be more positive I'll definitely stick with it, now I just feel picking a all-in-one is not the right choice.