Re:Exponential Smoothing: faster and more accurate than Neur...
by maxmc on Wednesday August 17, 2022 @08:05PM
(#32503495)
This comparison was not intended as an academic paper. However, we are confident that predicting and evaluating performance over more than 55k series from the M international forecasting competitions (standard benchmarks in the field) is a good start. After corresponding with the author, we included additional data sets from the electricity domain like Ercot and ETTM2. The dataset that did not finish training shows that Neuralprophet simply does not scale. (We let the training run for over 73 hours with 96 CPUs -as NeuralProphet implementation is restricted to CPU-, and we canceled it after investing 288 USD in it) Suggestions to strengthen our experiments (without spending hundreds of dollars) are highly welcomed.
Regarding the hyperparameter selection, we went beyond the original NeuralProphet paper, tunning and actively trying to help improve its performance. We even made a PR fixing NP bugs in the process.
Re:Exponential Smoothing: faster and more accurate than Neur...
by predict_addict on Thursday August 18, 2022 @02:44AM
(#32506119)
This is ironic, because the Neural Prophet paper wasn’t reviewed either and didn’t benchmark it on anything other than Prophet.
Re:Exponential Smoothing: faster and more accurate than Neur...
by gillesjacobs on Wednesday August 17, 2022 @05:21PM
(#32501892)
While the results don't prove the superiority convincingly, it does seem that ETS is a good candidate as a first go-to in practical applications. "In practice, practice and theory are the same. In theory, they are not.”
Re:Exponential Smoothing: faster and more accurate than Neur...
by variaga on Wednesday August 17, 2022 @05:36PM
(#32502025)
In theory practice and theory are the same. In practice they are not.
Re:Exponential Smoothing: faster and more accurate than Neur...
by gillesjacobs on Wednesday August 17, 2022 @05:51PM
(#32502185)
HN pedantry ruins the fun of wordplay yet again.
Re:Exponential Smoothing: faster and more accurate than Neur...
by anon_123g987 on Wednesday August 17, 2022 @06:01PM
(#32502273)
In theory, his version is right. In practice, yours.
Re:Exponential Smoothing: faster and more accurate than Neur...
by lr1970 on Wednesday August 17, 2022 @09:12PM
(#32503990)
The difference between theory and practice is greater in theory than in practice.
Re:Exponential Smoothing: faster and more accurate than Neur...
by timy2shoes on Wednesday August 17, 2022 @09:30PM
(#32504115)
In theory there is no difference between theory and practice. In practice there is.
- Yogi Berra
Re:Exponential Smoothing: faster and more accurate than Neur...
by DoctorOetker on Thursday August 18, 2022 @03:02AM
(#32506255)
"Sometimes malpractice is just malpractice"
- Sigmund Freud
- No fair hyperparametrization for Neural Prophet. They mention multiple times they used default hyperparams or ad-hoc example hyperparams.
- 3/4 benchmark datasets (one they didn't finish training) where ETS outperforms is not strong evidence of all-round robustness. Benchmarks like SuperGlue for NLP combine 10 completely different tasks with more subtasks to assess language model performance. And even SuperGlue is not uncontroversial.