News for hackers. Stuff that matters.
Re:Machine Learning by jey on Wednesday May 17, 2017 @04:29AM (#14356575)
"Just stir the pile until [the answers] start looking right" is actually a pretty decent description of gradient descent.
Re:Machine Learning by james_a_craig on Wednesday May 17, 2017 @05:10AM (#14356754)
It's clearly simulated annealing. You stir less as you get tired. :)
Re:Machine Learning by randomsearch on Wednesday May 17, 2017 @02:05PM (#14361044)
Simulated annealing also exploits gradient information, so it is not equivalent to random search, although it is a stochastic algorithm.
Re:Machine Learning by dmitriy_ko on Wednesday May 17, 2017 @04:42AM (#14356621)
With gradient descent you are walking in the direction of negative gradient. Stirring the pile implies that you are walking randomly from point to point.
Re:Machine Learning by TeMPOraL on Wednesday May 17, 2017 @05:43AM (#14356905)
No, by stirring you're just making the pile to move, and things descend down the gradient of gravity. :).
Re:Machine Learning by steanne on Wednesday May 17, 2017 @10:40AM (#14358850)
The little things moreso. Brazil nut effect!
Re:Machine Learning by yakult on Wednesday May 17, 2017 @05:51AM (#14356942)
When you stir, you're stirring in a pattern, probably a circular one. Plus you're probably not paying too much attention to the shape of the pile, so it's gradient-free. It's pattern search. Except it's only approximately a circular pattern. There are random variations. Stochastic pattern search.
Re:Machine Learning by andai on Wednesday May 17, 2017 @05:11AM (#14356760)
So more like a genetic algorithm then?
Re:Machine Learning by randomsearch on Wednesday May 17, 2017 @02:06PM (#14361055)
GAs also exploit structure in the search space.

To see this most clearly, read up on estimation of distribution algorithms (EDAs), which generalise the ways in which GAs work.

Re:Machine Learning by netheril96 on Wednesday May 17, 2017 @11:20AM (#14359297)
It is more like tuning the hyper-parameters, which is guided by anything but logic.
Re:Machine Learning by jrdt on Wednesday May 17, 2017 @05:24AM (#14356817)
It actually seems almost like the definition of RANSAC.

https://en.wikipedia.org/wiki/Random_sample_consensus

Re:Machine Learning by randomsearch on Wednesday May 17, 2017 @02:04PM (#14361040)
Gradient descent is not random search, so this is an inaccurate description. It works by exploiting the gradient in a structured landscape of solutions.
Re:Machine Learning by denzil_correa on Wednesday May 17, 2017 @04:41AM (#14356615)
I'm not so sure. Gradient Descent refers to reaching the best solution available. Xkcd seems to be referring to more like the Bonferroni principle where you are looking for patterns without a hypothesis and post facto justifying it.
Re:Machine Learning by YeGoblynQueenne on Wednesday May 17, 2017 @08:40AM (#14357811)
I think the point is about how data manipulation is about 80% of machine learning work. If an algorithm is giving you crap results for some data set, once you've fiddled with its hyperparameters through cv and so on there's not much you can do besides data manipulations like PCA, ICA and the like, to try and get a better result. Most algorithms work pretty badly with raw data.

I'm guessing that, in the sciences, that is a big no-no. Imagine if doctors, seeing that a new drug being trialled is failing to cure a disease, simply started chucking out the sick subjects until all the ones that were left where healthy, declaring the sick ones to be "noise" and the trial a success. Somehow, I don't think that would fly...

Re:Machine Learning by randomsearch on Wednesday May 17, 2017 @02:09PM (#14361095)
This is confusing "cherry picking" (selecting some data and discarding other data) with "data transformation". Transformations are applied for perfectly good reasons, such as dimensionality reduction, or to enable the subsequent application of certain statistical tests that rely on assumptions not valid for the data in its original form.
Re:Machine Learning by denzil_correa on Wednesday May 17, 2017 @10:24AM (#14358692)
> I think the point is about how data manipulation is about 80% of machine learning work. If an algorithm is giving you crap results for some data set, once you've fiddled with its hyperparameters through cv and so on there's not much you can do besides data manipulations like PCA, ICA and the like, to try and get a better result.

Data manipulation is different from data transformation. Manipulation changes the nature of data, transformation does not change the nature of data. PCA is data transformation, not data manipulation.

Re:Machine Learning by YeGoblynQueenne on Wednesday May 17, 2017 @12:34PM (#14360103)
Tell me what you mean by "the nature of data" and I'll tell you if I agree with your definition. I don't see why the dimensionality of a dataset is not its "nature" for example.
Re:Machine Learning by ardivekar on Wednesday May 17, 2017 @07:49AM (#14357510)
AKA half the work done in Machine Learning nowadays.
Re:Machine Learning by randomsearch on Wednesday May 17, 2017 @02:07PM (#14361076)
To be precise: the best solution reachable from a given starting point, which may be a local minima rather than a global minima.
Re:Machine Learning by nabla9 on Wednesday May 17, 2017 @08:59AM (#14357952)
(iterative) stochastic gradient descent to be specific. Data is stirred for each pass to prevent cycles.