Ideally the model would insert an API call into its output to fetch the contacts first, then make the call.

The scary part is asking the LLM to read your messages. Imagine someone texts you an LLM jailbreak. Siri might not be Siri anymore after it finishes reading.