Re:LLMs, RAG, & the missing storage layer for AI
by jamesblonde on Friday September 08, 2023 @01:26AM
(#37429702)
Why would your projection be this - SELECT LLM("{user_question}", ?
You can train a small llm on your private data to map the user question to tables in your db.
Then Just select with a limit ( or time bounded). The feature store is just another operational store that could have relevant data for the query.
Re:LLMs, RAG, & the missing storage layer for AI
by fkyoureadthedoc on Friday September 08, 2023 @09:36AM
(#37433321)
> You can train a small llm on your private data to map the user question to tables in your db.
Can you? You've personally done this? Deployed it to production at some kind of non trivial scale and it's working well? I'm not aware of any "small llm" that approaches the quality of gpt-3.5.
Re:LLMs, RAG, & the missing storage layer for AI
by philipodonnell on Friday September 08, 2023 @12:19PM
(#37435664)
This is called Text2SQL or NL2SQL, it’s a surprisingly difficult problem even with RAG and GPT4 as soon as the query is non trivial, especially if there are semantic differences between the question and the db schema.
1. Will that query look like this:
2. How will a feature store, like Hopsworks, help in this app?Shameless self-plug: We are building EvaDB [1], a query engine for shipping fast AI-powered apps with SQL. Would love to exchange notes on such apps if you're up for it!
[1] https://github.com/georgia-tech-db/evadb