It's very unlikely that Google employees are going to be representative of Google's users as a whole (just look at the startups building niche products for Bay Area techies). Ditto if you only looked at data of users that opted in (maybe they're highly unrepresentative).

That means the heuristics/models that get built would be a worse fit for you, as a user, although it sounds like that's a tradeoff that you'd like.

Exactly. Google is saying they "need the cloud" to make the prediction/action. They also require as grist for their mill: my data.

I would much prefer some basic heuristics executed on-device using my data in a secured processing state. The hueristic-building could be in the cloud, but on datasets other than mine, and let me tweak those dials!

Hell, I'd even pay for this feature.