Re:Machine Learning Is the New Statistics
by eli_gottlieb on Monday October 17, 2016 @09:59AM
(#12725033)
>Huge data sets today render most statistical tests useless since no hypothesis is strictly true given enough precision, and the focus on eking the most out of every data point is overkill.
Mind, huge data sets and huge amounts of computing power also make the use of Bayesian methods feasible, which don't have the NHST problem that anything is significant with enough data-points.
To make the case that ML - stats isn't empty, consider neural networks and singular value decomposition (Netflix winners used SVD): both are wildly successful ML techniques – and neither is remotely statistically sound. Their correctness is at best heuristic, yet, partly because they can be efficiently applied to huge amounts of data – far more than classical statistical methods can handle – they are very effective.
Statistics has traditionally focused on making the most of a modest amount of data – because in previous eras, data was the limiting factor. Huge data sets today render most statistical tests useless since no hypothesis is strictly true given enough precision, and the focus on eking the most out of every data point is overkill. It turns out that a naive, imprecise method with a huge amount of data is often more effective than a sophisticated method with far less data. As ML gets better at building statistically sound models and stats gets better at scaling, the two disciplines are converging slowly – but we are still very far away from inclusion in either direction.