> your AI should ultimately know your favorite restaurant, your girlfriend’s name, but also your health record and everything else you might not always feel comfortable sharing with the world at large.

No. It should know what the user wants it to know. Which may or may not be those things.