The big question is whether we can build systems that communicate, but don't create vast disparities of information between overminds and everybody else. At this point the handful of information slurping mega-corporations have a huge information advantage compared with the average person. If nothing changes, this is the end of the humanistic world order based on the primacy of the individual brain.
I suspect is usually a mixture. For example Google Translate runs locally if you download a language pair, yet the training is not local. Google Gboard Swipe runs locally: in airplane mode I just added a new word to the dictionary and swipe recognised it, also contact names. However Google sends swipe information to a central location to improve the service...
I think there is a battle between two competing AI philosophies. One philosophy thinks AI should be central: all devices need to talk to the overmind for learning. The other philosophy thinks AI should be local: the device should learn on its own, kind of like a biological agent. I think the 'biological agent' form of AI is interesting when it comes to privacy of information. Only you and the device have the data.
The future of AI should be interesting with these two philosophies in mind.