That is a staggering rate of increase. I can see a future where this is less centralized; learning could happen in "phases" where a local device improves its model given local data and reports back something centrally that can be combined and used to train a shared model.

This requires hardware to be miniaturized as non-ML compute has been and when that does happen we'll have the learnings from the current edge computing push. In the mean time I've excited to see what developments are made on both the hardware and software side.