Re:Statistical vs. Deep Learning forecasting methods
by zone411 on Thursday December 01, 2022 @01:33PM
(#33820485)
Every researcher would love to include error bars but it's a matter of limited computing resources at universities. Unless you're training on a tiny dataset like MNIST, these training runs get expensive. Also, unless you parallelize from the start and risk wasting a lot of resources if something goes wrong, it could take longer to get the results.
[ Parent
| on HN ]
Re:Statistical vs. Deep Learning forecasting methods
by PaulHoule on Thursday December 01, 2022 @02:04PM
(#33820985)
Using bootstrap and/or repeated runs is a great way to get error bars but there are low cost ways to do it.
For instance they estimate error bars on public opinion polls based on simple formulas and not redoing the poll a large number of times.
Re:Statistical vs. Deep Learning forecasting methods
by nequo on Thursday December 01, 2022 @02:36PM
(#33821511)
If you don't have an analytical expression for your asymptotic variance, you do have to use bootstrap though.
For public opinion polls, the estimator is simple (i.e., a sample mean), so we have an analytical expression for its asymptotic variance.
Re:Statistical vs. Deep Learning forecasting methods
by time_to_smile on Thursday December 01, 2022 @02:41PM
(#33821583)
Simple formulas only work because the models themselves for those polls are incredibly simple and adding a bit more complexity requires a lot of tools to compute these uncertainties (this is part of the reason you see probabilistic programming so popular for people doing non-trivial polling work).
There are no simple approximations for a range of even slightly complex models. Even some nice computational tricks like the Laplace approximation don't work on models with high numbers of parameters (since you need to compute the diagonal of the Hessian).
A good overview of the situation is covered in Efron & Hastie's "Computer Age Statistical Inference".