If I tell it to "shtudown", it said "did you mean 'shutdown'?
There is an AI failure in distinguish "mispronounced command I know" from "command I don't know".
Basic intent recognition models are trained to produce a single neuron per intent as the output, which makes it pretty easy to use the activation levels of the output to decide whether to perform an action, confirm an action, or ask for clarification. You just need to check if the certainty is below a certain threshold.
With ChatGPT you'd have to encode the intents as text of some kind (JSON?) and hope that it doesn't just hallucinate an intent that your APIs don't have when it's faced with ambiguous input. You could probably have hallucinated intents map to a decent-sounding error message, but that feels more brittle to me than the existing approaches.
Ask it to write an SQL query and it will.
Give it upfront context and it will be able to produce API commands.
The voice recognition can trigger chatgpt API 'create API command for the following text's.
The practical problem is cost and hardware.
But this is closer than we ever were.
A much safer approach would be to have it produce a JSON object that is interpreted by the wrapping code. This allows you to inspect the interpreting code and provide guarantees about what can and cannot be done by users.
But neither execution model solves my main point: someone still has to write the interpreter (or the library functions if you go the dangerous route). Someone has to think ahead and guess what the user is going to try to do and provide APIs that allow that. And that work is most of the work: our existing intent recognition models are actually not bad at all at their job, the failings these voice assistants have are almost all to do with predicting the ways in which users will try to interact with the system.
The code generation and execution is trivially problematic. I don't need to try it out to know that if someone can coerce one of these bots to spit out its prompt, someone could coerce it to execute dangerous code if it's wired up to an interpreter.
As for the other part of my analysis, in order to empirically test it I would have to build a full-featured voice AI, release it to millions of users, and see if I was able to predict everything they tried to do. Given that my prediction is that even Apple can't do that well, I'm not sure why you think I would bother to try it out.
If an engineer told you that a bridge would collapse if built a certain way, you wouldn't insist on trying to build it anyway just to be sure. Most of the time in engineering you can't run a full-scale test, you have to make do with analyses. If you want to critique my analysis, I'm all ears, but "you haven't actually tested it" is not a critique.
All of my analysis is based on my usage of ChatGPT and my understanding of the underlying model. The hype is largely driven by people who don't understand how it actually works and think they're interacting with an artificial general intelligence.
It's a very impressive language model with a lot of applications, but with many fewer applications than the hype would suggest.
I'm hyped because I threw normal text (sentences) against it and it always did what I wanted and it didn't matter if it was bad English or bad German.
I didn't even try promt hacking or promt tuning because natural language already worked so good.
It's a tremendous good 'normal human' text interface.
ChatGPT can't do anything. Someone has to, somehow, wire it up to actual physical or digital controls. Intent recognition and slot filling still have to happen somewhere, and that means defining discrete intents and slots.
GPT may be better at mapping from speech to intents, but it can't magically interface with APIs that haven't been defined.
EDIT: To elaborate a bit, the problem in OP's story isn't caused by a bad language model, it's caused by no one at Apple thinking to define different "alert volume" and "media volume" intents. Current language models are plenty good enough to recognize the distinction, so simply adding ChatGPT won't be enough to make any feature work unless someone at Apple predicts the need and writes the interface.