I don't think this is something we should worry about. If you want better models use Google. If you want better privacy go with Apple. It's fantastic that we actually have a choice and don't all have to sign our privacy away or live in the dark ages.
AlphaGo is impressive no doubt but has DeepMind done anything really key to Google's bottom line yet? A lot of the really sweet stuff they do doesn't have immediate commercial utility that I can see. Apple might be waiting to strike once Google has found the killer app (remember they're never really first at anything and focus holistically on the product).
Apple is benefiting from other companies releasing their research. If everyone but Apple releases the community is nearly as good, and Apple gets the pick of external and internal research while not needing to give up any of their own ideas. I know they send people to conferences and it can be a bit weird talking to someone who won't tell you anything about what they do.
Regarding researchers, I don't know of any top trend setters who've joined but they do have some very good applied ML people through direct- or aqui-hires.
Tl;dr: Apple doing what they usually do and keeping their powder dry/free loading off other's work until they can execute the product.
To all factories, power plants, homes, etc.
Imagine you can cut the cooling/heating bill of 1/2 the industrial world by 40%.
The 200 MB figure quoted appears to refer only to the model stored locally on the phone. In my experience, 200 mb translates to a few million parameters in one or more sparse matrices.
The figure on the whiteboard in the background says "Hey Siri small". I take that to indicate the model that does feature extraction and prediction for some queries, such as "set a timer for 20 minutes", while there is a larger, more general model for other use cases in the cloud.
https://backchannel.com/how-google-is-remaking-itself-as-a-m...
Apple's PR is notorious for cracking the whip, which means that the "inside story", if they give it to you, comes with a warning to the journalist to behave and be nice. Levy's piece is generous with flattery and cautious with criticism. He quotes Kaplan and Etzioni high and briefly in the piece, and spends the rest of it refuting them. Apple will give him another inside story down the road.
Apple has a big question to resolve for itself about the tools it's going to use to develop this. It can't go with Tensorflow, because TF is from Google. It's kind of at another turning point, like the one in the early 90s when it needed it's own operating system and Jobs convinced them to buy next and use what would become OSX.[0]
The most pointed question to ask is: What are they doing that's new? The use cases in the Levy story are neat, and I'm sure Apple is executing well, but they don't take my breath away. None of those applications make me think Apple is actually on the cutting edge. There's no mention of reinforcement learning, for example; there is no AlphaGo moment so far where the discipline leaps 10 years ahead. And the deeper question is: Is Apple's AI campaign impelled by the same vision that clearly drives Demis Hassabis and Larry Page?
We see what's new at Google by reading DeepMind and Google Brain papers. Everyone else is letting their AI people publish, which is a huge recruiting draw and leads to stronger teams. Who, among the top researchers, has joined Apple? Did they do it secretly? (This is plausible, and if someone knows the answer, please say...) The Turi team is strong, yes, but can they match DeepMind? If Apple hasn't built that team yet, what are they doing to change their approach?
Another key distinction between Apple and Google, which Levy points out, is their approach to data. Google crowdsources the gathering of data and sells it to advertisers; Apple is so strict about privacy that it doesn't even let itself see your data, let alone anyone else. I support Apple's stance, but I worry that this will have repercussions on the size and accuracy of the models it is able to build.
> “We keep some of the most sensitive things where the ML is occurring entirely local to the device,” Federighi says.
Apple says it's keeping the important data, and therefore the processing of that data, on the phone. Great, but you need many GPUs to train a large model in a reasonable amount of time, and you simply can't do that on a phone. Not yet. It's done in the cloud and on proprietary racks. So when he says they're keeping it on the phone, does he mean that some other encrypted form of it is shared on the cloud using differential privacy? Curious...
> "How big is this brain, the dynamic cache that enables machine learning on the iPhone? Somewhat to my surprise when I asked Apple, it provided the information: about 200 megabytes.."
Google's building models with billions of parameters that require much more than 200MB, and that are really, really good at scoring data. I have to believe either that a) Apple is not telling us everything, or b) they haven't figured out a way to bring their customers the most powerful AI yet. (And the answer could very well be c) that I don't understand what's going on...)
[0] If they have a JVM stack, they should consider ours: http://deeplearning4j.org/