> but no way I'm letting some LLM book a hotel that might not exist or get me car insurance from StratesFrarm Unsurance.

Ah, scammers targeting your LLM assistant will certainly be a thing, this really sounds like the old "I bought the Eiffel tower" scams.

I'm not clear how this is different from scammers targeting humans? If anything, I'd expect it to be easier to set up the LLM agent to have secure/cryptographical validation of the providers, to prevent against the sorts of traps that humans fall into.

In the simplest case, we could just have trusted authorities provide lists of verified booking partners, same as e.g. Google Flights does now.