In short, humans kinda suck. LLMs also kinda suck, but faster than humans.
Agreed, but that's part of my critique. These systems are written by humans and are trained with barely curated data generated by humans. There is the concept of emergence, but I'm not sure how emergence suddenly fixes a terrible foundation full of biases and errors.
It's correct enough to pass the bar exam, medical exams, it scores 90-93 percentile on the SAT. This is way more complex and efficient than what you make it to be imo.
So is Google Search, and we had that for a long time now. Is a slightly different and more verbose UI really a game changer?
(Let's ignore the fact that Google Search is broken from all the SEO spam and monetization. Especially when we have no evidence that ChatGPT is any more resitant to this than Google was.)
I hooked GPT-4 up to a shell and asked it to use the GPT-3.5-turbo completions API (not in the dataset), and it successfully did it through trial and error with curl and error messages. This example is of course not something you would actually do regularly, but rather shows that you don't need a lot of context for it to do useful things right now. With a complete explanation of the OpenAI endpoints, it would most likely make the request perfectly on the first try.
'only answer the following questions and use these API endpoints doing the task'.
Now you have a multi lingual human Isabel Interface.
Have you seen the Wolfram alpha examples?
Think of it as a natural language interface to anything that has an API.