IMO a semantic layer makes it so easy to query that natural language querying is often not beneficial, even for business users. I take a hybrid approach of the semantic layer solving >95% of use cases and custom SQL as needed in backend processing.
Wrt llmindex, the new tools like vector dbs & llmindex have been interesting. Fine-ish for our self-hosted, but most start getting tricky when a multi-tenant SaaS where we lower costs for teams and users by sharing infra yet need sharing boundaries.
We're SOC2 compliant which has been enough for many companies to get comfortable with security / data handling practices.
I would open source it if I had decent experience with OSS.
If anyone with OSS maintainer experience wants to help with that - ping me.
Still, would you mind sharing the use case and the outcome? I have a couple of clients interested in similar solutions, but so far the potential outcome of this approach doesn't look promising.
We building a natural language chat interface for structured data. It can handle small csv files to large databases(500+ tables) seamlessly.
1. Pulling in your schema information and structuring it in a way LLM's can reason about it
2. Pulling in your prior query history against the database to understand how you actually use your data (e.g. what JOIN's are common, what tables are used most frequently, etc.)
3. Adding context from other tools you may be using (e.g. we can pull in metadata and tests from your dbt project)
We also have a Slackbot you can add to your #urgent-data-requests channel. If you @Definite in a thread, it'll parse out messages that can be converted to SQL tasks and return the answer from your database.
You could certainly build this yourself with (or without) LlamaIndex, but it's still quite a bit of work to set up.