News for hackers. Stuff that matters.
Re:Google's AutoML: Cutting Through the Hype by halflings on Wednesday August 01, 2018 @08:16PM (#17668246)
"In evaluating Google’s claims, it’s valuable to keep in mind Google has a vested financial interest in convincing us that the key to effective use of deep learning is more computational power, because this is an area where they clearly beat the rest of us. If true, we may all need to purchase Google products. On its own, this doesn’t mean that Google’s claims are false, but it’s good be aware of what financial motivations could underlie their statements."

Excuse the snark, but:

"In evaluating FastAI's claims, it’s valuable to keep in mind FastAI has a vested financial interest in convincing us that the key to effective use of deep learning is more machine learning experts, because ML education is their business model. If true, we may all need to purchase FastAI courses. On its own, this doesn’t mean that FastAI’s claims are false, but it’s good be aware of what financial motivations could underlie their statements."

I can think of thousands of ways that Google could increase the computational power required that would be much easier than the AutoML effort (for ex. simply recommending ridiculously deep models that take 100x to train without giving higher performance). They are putting so much effort into AutoML because it just works. A lot of the things included in later parts of this series are very useful (learning rate search, etc.) and decrease the computational power required, but most people just want to drop a dataset and pick the type of model (say, multiclass image classification) and leave the rest for machines to optimize.

Re:Google's AutoML: Cutting Through the Hype by citrablue on Wednesday August 01, 2018 @09:37PM (#17668633)
Your FastAI criticism isn't really accurate, as FastAI is free online courseware. I suppose you could claim that jhoward makes his living from his personal brand, and FastAI helps that... but that's a bit of a stretch.

It would be much more valid if they were a business, rather than an open source software and free course solution. Even as a competitor, it's probably useful criticism to get a negative perspective from a competitor.

Re:Google's AutoML: Cutting Through the Hype by voidray on Wednesday August 01, 2018 @09:47PM (#17668672)
> most people just want to drop a dataset and pick the type of model (say, multiclass image classification) and leave the rest for machines to optimize.

I think the disconnect here is that you can reuse existing architectures and get state-of-the-art performance without running something like AutoML. It's not clear that creating a bespoke architecture for your specific problem is always better, let alone always a good use of your resources.

Re:Google's AutoML: Cutting Through the Hype by halflings on Thursday August 02, 2018 @12:53AM (#17669300)
Frankly, I don't think this is obvious. Case in point: sequential models (granted, probably not yet included in the first batch of AutoML). There are so many ways to model the problem that tweaking each of these ways, in a way that makes sense for your dataset, takes a lot of work.

I've built models that worked fairly OK, only to have a colleague build a separate model that had +10% prAUC by virtue of adding some additional mechanism (say, attention, a different RNN cell, more units, etc.).

I'll also say that this is aimed towards people that are unfamiliar with ML and would have trouble finding and re-implementing the state-of-the-art in their specific field.

Re:Google's AutoML: Cutting Through the Hype by halflings on Thursday August 02, 2018 @12:49AM (#17669291)
Sorry for the tone here (unusually snarky comment).

Arguably, my point was that this argument, biasing your analysis by hidden motives that cannot be disproven, is invalid.

But clearly, I was wrong because fast.ai's courses are indeed free so the comparison doesn't stand.

I'll also add that fast.ai has great ML courses that go deep into details, and focus on practical, applicable, techniques. My comment was specifically about the argument that Google is doing this just to increase Cloud revenues.

Re:Google's AutoML: Cutting Through the Hype by amrrs on Wednesday August 01, 2018 @11:09PM (#17668992)
Ignoring the fact that Auto ML can't have an impact is how Nokia and Blackberry laughed at iPhone that "huh! who'll use a full-touch phone". Data Scientists by profession are really insecure about this fact of Auto ML hence making a claim that can free up their mind.

Data Scientists in many companies have become close to what SQL Job or any web script job runners used to do and then CRON jobs came into pic and that's what Auto ML will do here with a a set of sequential steps replacing mundance ML activities.

Re:Google's AutoML: Cutting Through the Hype by YeGoblynQueenne on Thursday August 02, 2018 @06:42AM (#17670478)
>> They are putting so much effort into AutoML because it just works.

It just works, provided your dataset can be solved well enough given AutoML's pretrained model. Which is not at all guaranteed.

Re:Google's AutoML: Cutting Through the Hype by tntn on Wednesday August 01, 2018 @10:18PM (#17668806)
Wow, you sure got them good. Thanks for rooting out the financial motivations of fast.ai, a nonprofit research lab that publishes courses online for free.

/s

Re:Google's AutoML: Cutting Through the Hype by pwaai on Wednesday August 01, 2018 @11:18PM (#17669023)
Some people will be cynical for the sake of it.