> 80% of people just want the ability to write a sentence and have it run the correct SQL on their data warehouse.
Even this is IMO too ambitious (due to edge cases, and multi-shot prompting being basically required, which kind of defeats the purpose). Right now, I'm working on a product that can just do simple OS stuff (and even that is quite challenging); e.g. "replace all instances of `David` with `Bob` in all text files in this folder".
I think you are right -- eventually we will see an ecosystem pop up that uses LLMs in unique ways that are only possible with LLMs.
But in the meantime people are using LLMs to shortcut what used to be long processes done by humans, and frankly, there is a lot of value in that.