How do you know the LLM is going to do what you confirmed?

There's a fundamental tension here: either you limit the LLM to a set of fixed actions that a user can individually understand and confirm. Or you let it figure out what to do and how to do it given a higher-level goal.

In the first case, it's limited and not really better than a well-designed site or app. In the second it's powerful but can run amok.

E.g., did it really fill out that four page order form for that e-bike you asked it to order? Maybe it uses the debit card instead of your credit card and your checking account is overdrawn. Maybe it sets the delivery address to your brother's address. Maybe it orders two bikes or 10 or the wrong model or wrong size.

OR

It asks/confirms each step of the way so there's little chance for mistakes.

This is an inherent problem with any kind of delegation. You either micro manage the details or trust your agent to get them right.

The whole system isn't an LLM. At the point they are asking for confirmation, they've already parsed the required information and handed it off to normal code. It's not going to change again.

Ultimately, an LLM only has the capability to take a string as input and output another string. Any other functionality (Uber, search, travel, etc) has to be manually programmed in via APIs. For the foreseeable future, a fixed set of actions is the only way to do this reliably and is what this device does so talking about the second case of an LLM run amok is a moot point.

I also don't understand why they call it LLM. it isn't.

I think they call it a Large Action Model not an LLM.

They can call it whatever they want but it’s essentially IFTT + DOM scraping /automation (“teaching” lol) for places without an official API with a super thin AI veneer. Mostly for suggestions.

Funny thing that the keynote only demoed “Confirm” and never “Correct”/“Change this”, etc.

Absolutely and that will be a major usability problem when LLMs get things wrong as they so often do

At one point he asked for a more relaxed version of the initial vacation plans.

May be because "change this" is vague and more of a broad function and may need more clarifications. Users may have more clearer commands, like "cancel this", "I don't like this, provide an alternative" or "can you go back one step?" etc. User can just speak those requests out loud since device is listening.