Machine learning without centralized training data

Re:Machine learning without centralized training data by nostrademons on Thursday April 06, 2017 @08:03PM (#14055864)

This is one of those announcements that seems unremarkable on read-through but could be industry-changing in a decade. The driving force between consolidation & monopoly in the tech industry is that bigger firms with more data have an advantage over smaller firms because they can deliver features (often using machine-learning) that users want and small startups or individuals simply cannot implement. This, in theory, provides a way for users to maintain control of their data while granting permission for machine-learning algorithms to inspect it and "phone home" with an improved model, without revealing the individual data. Couple it with a P2P protocol and a good on-device UI platform and you could in theory construct something similar to the WWW, with data stored locally, but with all the convenience features of centralized cloud-based servers.

[ Parent | on HN ]

Re:Machine learning without centralized training data by jd20 on Thursday April 06, 2017 @09:08PM (#14056213)

I think you could be spot on: there are new applications emerging in deep learning, like self-driving vehicles, where you have a powerful need for mountains of data to train complex models, yet a logistics problem in how to aggregate that data in a single place (I'm making this up, but imagine a car collecting 7 streams of 4K video at 60 fps). I really see a growing need for these types of distributed training models.

[ Parent | on HN ]

Re:Machine learning without centralized training data by Fricken on Friday April 07, 2017 @01:26AM (#14057361)

I'm a bit confused. Doesn't the data need to be cleaned up, annotated, or labelled before it can be used to train? How would this work if the data doesn't leave the local device?

[ Parent | on HN ]

Re:Machine learning without centralized training data by jd20 on Friday April 07, 2017 @03:01AM (#14057712)

Sounded like the label for training is the user's action (whether they chose the suggestion provided or not). Then they collect the model updates in bulk from millions of users, that's far more useful for training purposes than a small very well curated data set.

[ Parent | on HN ]

Re:Machine learning without centralized training data by vitohuang on Friday April 07, 2017 @02:22AM (#14057561)

It did mentioned a local version of mini TensorFlow, I would guess the data is cleaned up on local devices and only updates are send back to the cloud.

[ Parent | on HN ]

Re:Machine learning without centralized training data by amelius on Friday April 07, 2017 @03:55AM (#14057946)

There is a branch called unsupervised learning, where no labeling is needed.

[ Parent | on HN ]

Re:Machine learning without centralized training data by ska on Friday April 07, 2017 @10:08AM (#14059729)

There is, but supervised and unsupervised learning mostly don't solve the same problems - so this doesn't necessarily help

[ Parent | on HN ]

Re:Machine learning without centralized training data by bryanrasmussen on Friday April 07, 2017 @05:00AM (#14058238)

as I understand what I read the data does leave, but just the users updates - sort of like only your changes to a git a branch are sent when you push.

[ Parent | on HN ]

Re:Machine learning without centralized training data by XJOKOLAT on Friday April 07, 2017 @05:14AM (#14058293)

Not knowing enough about this, my first impression is:

How does Federated Learning cope with ... FAKE NEWS! (for example).

[ Parent | on HN ]

Re:Machine learning without centralized training data by brk on Friday April 07, 2017 @10:46AM (#14060088)

It is mostly applicable for taking data sourced at an endpoint (e.g. Mobile phone) and running what is essentially a refinement to the learning.

A key component is that to analyze the data in the cloud for the same refinement would mean sending the data to the cloud, which the user may not want, and may also be bandwidth intensive.

For fake news, the data is already in the cloud, being pushed down to the users device. A user could mark something as 'fake' (via explicit action, or possibly inaction), and that 'mark' is uploaded and the data is analyzed by central compute for refinement. To be clear, that is NOT what this paper is about, I am saying fake news would be a bad use, because the data is not being sourced by the user, only viewed and possibly marked.

A better example than the gboard gesture learning refinements might be in the form of an app that acts as a dashcam (leaving aside the kludgeyness of that). The user could mark things like proper recognition of street signs, traffic lights, or brands/models of vehicles. The phone would then analyze those classification, compute a diff to the algorithm, send just that diff to the cloud (summarizing a lot). Multiply that by 1,000,000 users, and now you have a refined data set that did not require sending 100,000,000,000 images to the cloud for analysis.

[ Parent | on HN ]

Re:Machine learning without centralized training data by marchenko on Friday April 07, 2017 @06:07AM (#14058493)

For federated learning to truly reduce incumbent advantage, won't newcomers still have to develop a platform attractive enough to entice device users to download their model and consent to data inspection? Or is the idea that Google or another large provider will be the point of entry for various algorithms to connect to the datastream, as a sort of cloud service? How many models can co-exist on one device?

[ Parent | on HN ]

Re:Machine learning without centralized training data by brianorwhatever on Thursday April 06, 2017 @08:33PM (#14056035)

Which is why I am surprised seeing this come from Google. Everyone has already admitted they are fine sending all of their data to them which benefits them greatly.

[ Parent | on HN ]

Re:Machine learning without centralized training data by nostrademons on Thursday April 06, 2017 @08:42PM (#14056086)

I think this may be a Xerox Alto, IBM PC, or Sun Java moment. In the short term I can see a clear benefit to Google for this. They want to get machine-learning into more aspects of the Android mobile experience, Android customers are justifiably paranoid of sending things like every keystroke on the device back to Google's cloud, and so this gives them a privacy-acceptable way to deliver the features that will make them more competitive in a new market. Remember that the vast majority of Google employees honestly want to do what's best for the user, not what preserves Google's monopoly.

[ Parent | on HN ]

Re:Machine learning without centralized training data by Jedi72 on Thursday April 06, 2017 @09:26PM (#14056300)

The vast majority of people in any organisation are good people who want what's best in the sense of the greater good - that does not prevent organisations from doing bad things.

[ Parent | on HN ]

Re:Machine learning without centralized training data by thr0waway1239 on Friday April 07, 2017 @02:29AM (#14057586)

>> The vast majority of people in any organisation are good people who want what's best in the sense of the greater good

No. The vast majority of people in any organisation are timid folks who want what's best in the sense of the greater good, unless the greater good involves any courage on their part. These people are congenial, but don't confuse those with good people.

[ Parent | on HN ]

Re:Machine learning without centralized training data by jamesblonde on Friday April 07, 2017 @03:38AM (#14057856)

I have never heard it put so pithily, nice.

[ Parent | on HN ]

Re:Machine learning without centralized training data by pooper on Friday April 07, 2017 @02:22AM (#14057560)

I don't hold the leadership at Google with the same contempt as I hold the leadership at say Verizon, GE, Ford, or News Corp (WSJ). I'm sure given enough time all corporations are categorically evil but Google isn't there yet.

[ Parent | on HN ]

Re:Machine learning without centralized training data by memmcgee on Friday April 07, 2017 @04:25AM (#14058096)

Oh god. Has YouTube drama escaped to Hacker News?

[ Parent | on HN ]

Re:Machine learning without centralized training data by nintendo1889 on Thursday April 06, 2017 @08:59PM (#14056166)

Google's internal training emphasizes to do the right thing and compete fairly%, going so far as to not use terms in PR or even internal email such as 'crush the competition' 'dominate' 'destroy', and always doing what's good for the user, rather than bad for the competition.

% and often mentions competition/monopoly laws

[ Parent | on HN ]

Re:Machine learning without centralized training data by lern_too_spel on Friday April 07, 2017 @12:45AM (#14057187)

There's nothing altruistic about that. Emails with those words will cause problems for the legal department when they come up in discovery during antitrust litigation.

[ Parent | on HN ]

Re:Machine learning without centralized training data by brain5ide on Friday April 07, 2017 @07:46AM (#14058876)

Wittgenstein disagrees.

[ Parent | on HN ]

Re:Machine learning without centralized training data by lern_too_spel on Friday April 07, 2017 @06:30PM (#14063983)

Google forcing their employees to go through training to avoid bribery, sexual harassment, and antitrust problems for the company is not due to anything other than saving the company money. To be pedantic, the disagreement with GGP is not whether the actions are altruistic but whether the actions were done out of altruism.

[ Parent | on HN ]

Re:Machine learning without centralized training data by backpropaganda on Friday April 07, 2017 @04:30PM (#14063252)

It's easy being altruistic when you're the clear leader and have a comfortable margin. Doesn't give me any comfort knowing how benevolent Google is presently.

[ Parent | on HN ]

Re:Machine learning without centralized training data by samsamoa on Friday April 07, 2017 @12:00AM (#14057014)

True, but there are some kinds of data that people are still uncomfortable sending to Google. Medical data is considered especially private, and aggregation of medical data is a huge obstacle to improving treatment using ML. I think this could be really huge in that space.

[ Parent | on HN ]

Re:Machine learning without centralized training data by dekhn on Friday April 07, 2017 @10:23AM (#14059860)

There is a movement to change consent forms (which patients sign as they enter a medical system) to permit larger sharing of medical data outside of its direct use. IE, you are offered an opt-in to permit your data to be analyzed beyond an individual visit, possibly for medical issues completely unrelated to any immediate medical problem. The consent forms are transparent, and opt-in- the health consumer is informed what their data will be used for, and they explicitly have to say it's OK (blanket consent, can be revoked).

I think this is a win, because the consumer has the choice, and if enough people do it ,the resulting aggregated datasets will have exceptional power to help solve global medical problems.

[ Parent | on HN ]

Re:Machine learning without centralized training data by sdiepend on Friday April 07, 2017 @04:36AM (#14058136)

I'd say they reason they want your data is to make money, if they can do so without sending your data to their datacenters, why not?

If a goverment wants more power it makes sense to be able to read and store the data in their datacenter.

[ Parent | on HN ]

Re:Machine learning without centralized training data by RobertoG on Friday April 07, 2017 @03:44AM (#14057887)

It's not training a very computation intensive activity?

Maybe Google could be interested in move some of the processing load to the users devices.

[ Parent | on HN ]

Re:Machine learning without centralized training data by blackkettle on Friday April 07, 2017 @08:18AM (#14059003)

This maybe also dovetails with deep mind (?) or openai (?) article yesterday talking about adapting and updating general models with limited data for significantly different tasks.

[ Parent | on HN ]

Re:Machine learning without centralized training data by ganfortran on Thursday April 06, 2017 @09:25PM (#14056297)

Which is already the case like for 10 years. That is why no Startup ever challenge Google for their Search. It is basically the Matthew Effect but on Web Scale.

[ Parent | on HN ]

Re:Machine learning without centralized training data by memmcgee on Friday April 07, 2017 @04:39AM (#14058149)

I'm not too sure. There seems to be a spot for more niche search engines on the market. Take DuckDuckGo for example, it doesn't have that much of a market share, but it's a good option for people who care a lot about their privacy.

I would also consider WolframAlpha to be in a similar category as Google in the sense that they're "answer machines". I think there's a market opening for a search engine with the wisdom (knowing what you're looking for) of Google with the Intelligence of WolframAlpha (being able to draw curves that look like Darth Vader).

For example if you search "Darth Vader curve" in WA, you'll get this: http://m.wolframalpha.com/input/?i=Darth+Vader+curve&x=0&y=0

But if you can't remember exactly what it's called and search "Darth Vader graph" you'll just get redirected to the page for Darth Vader. Ideally, a WA would know what I'm looking for ("Darth Vader curve") and show it to me instead.

[ Parent | on HN ]

Re:Machine learning without centralized training data by justonepost on Friday April 07, 2017 @12:11AM (#14057061)

In a decade? Lol. This is being used today, right now, in many different forms. Trust me on that. Especially in the security field. The idea that Google just magically discovered this is cray cray. That being said, their contributions are hugely welcomed.

[ Parent | on HN ]