News for hackers. Stuff that matters.
Re:Ask HN: Lessons from Building a Fortune 500 RAG Chatbot (... by vlit20 on Friday March 21, 2025 @03:50AM (#43432856)
How did you measure success of the RAG solution beyond five-star user approvals? Are there any critical metrics that determine success or failure?
Re:Ask HN: Lessons from Building a Fortune 500 RAG Chatbot (... by tylersuard on Friday March 21, 2025 @02:34PM (#43439428)
We have tests in place to make sure every function, every search is working, otherwise things won't deploy. We do have a dashboard with usage etc., but I have not spent much time looking at it. I think we just count broad usage across the company as a success. And yes we consider one-stars and complaints to be a failure, but we get those less often, maybe once every 2 weeks, and it is almost always the user's fault. Does that answer your question?