Re:Exponential Smoothing: faster and more accurate than Neur...
by maxmc on Wednesday August 17, 2022 @08:17PM
(#32503589)
Figure 7 of the mentioned paper evaluates FB-Prophet in a extremely convenient environment of long horizon h in {30,60,90,120,150,180}. It is known that ETS and ARIMA models concatenate errors and degrade in performance with longer forecasting horizons. We have explored and offered solutions to these issues with the N-HiTS model specialized in long-horizon (https://arxiv.org/abs/2201.12886).
In recent years, FB-Prophet has gained a reputation for the poor quality of its predictions in many practical scenarios (short/medium term horizon) and its slow performance on bigger data sets; our ARIMA/ETS work and NeuralProphet confirmed those suspicions. The mentioned 92 percent improvements of the paper are restricted to h in {1,3,15,60} (https://arxiv.org/pdf/2111.15397.pdf).
This post's results are rather for short-horizon tasks, the same as NeuralProphet experiments. But we are confident that specialized tools like N-HiTS would outperform Prophet in long-horizon settings.
Re:Exponential Smoothing: faster and more accurate than Neur...
by disgruntledphd2 on Thursday August 18, 2022 @06:34AM
(#32507611)
Prophets original use case was for forecasting extremely long time series rather automatically. If that's not your use case, it may not be the right fit.
Re:Exponential Smoothing: faster and more accurate than Neur...
by fedegr on Wednesday August 17, 2022 @07:29PM
(#32503164)
I think the problem arises from the datasets used to evaluate the performance of the models. In the case of Prophet's paper, only one time series is used (The number of events created on Facebook). We can conclude from the results comparing AutoARIMA vs. Prophet (https://github.com/Nixtla/statsforecast/tree/main/experiment..., using the same datasets as in the ETS vs. NeuralProphet experiment) that ETS is also better than Prophet. Regarding NeuralProphet vs. Prophet, the results are not conclusive for these datasets.
Presumably this apparent non-transitivity is because of differences in each evaluation. If we fix the evaluation to the method used here, is it still the case that NeuralProphet outperforms Prophet (and therefore the claim that Prophet outperforms ETS is not correct)? Or is it that NeuralProphet does not outperform Prophet, but Prophet does outperform ETS?