Without knowing any more detail than “We got in touch with a few docs bot services and set up demos that were trained on our docs and blog posts.” it is hard to generalize to RAG + chat in general. I’ve had very good results with a custom setup that uses Claude Haiku to narrow down the set of relevant docs for a question and then 3.5 Sonnet to answer it. The corpus is on the small side, so no vector embeddings or even text search are required — the trick is understanding the different kinds of docs involved (OpenAPI schemas, hand-written guides) and writing code that abbreviates them in an appropriate way for the retrieval/narrowing step to work well.

I also manually tuned the system prompts to get the kind of answers I want and avoid the ones I don’t. I imagine off-the-shelf solutions are mostly lacking this customization, and they kind of can’t add it, because if they do, you’d be wondering what the value-add is and why you don’t build the same thing yourself in a couple of days. I’m sure techniques will improve, and it’s possible that turnkey solutions will be decent eventually.

I also think the distinction between supervised and unsupervised is misapplied here at the end, even accepting the colloquial use of a technical term. A docs tool powered by a bunch of hand-written documents and a custom system prompt, with a person asking questions of it — that doesn’t sound very unsupervised.