News for hackers. Stuff that matters.
Re:Moirai: A time series foundation model for universal fore... by wenc on Tuesday March 26, 2024 @01:04AM (#39824423)
They should sign up for the next Makridakis forecasting competition.

https://en.wikipedia.org/wiki/Makridakis_Competitions

Makridakis and Hibon reached the sad conclusion that "statistically sophisticated and complex methods do not necessarily provide more accurate forecasts than simpler ones."

Re:Moirai: A time series foundation model for universal fore... by hcarlens on Tuesday March 26, 2024 @04:55AM (#39825573)
That was true in the first Makridakis competition ("M1") in 1982, and possibly until M4 in 2018, but both M5 and M6 were won by what would generally be considered relatively sophisticated methods (e.g. LightGBM).

The Wikipedia article doesn't have that much detail on M5 or M6, but the M5 papers are in the International Journal of Forecasting[1] and M6 should be published later this year (there's already a preprint on arxiv [2]).

I recently spent some time looking into the history and results of the M competitions and had a chance to speak to Professor Makridakis about them, as well as the winners of each of the M6 competition tracks [3]. While the methods have become more sophisticated, some conclusions from M1 still seem to hold: in particular, that there is no overall "best" method, and that the winning method tends to be different for different types of data, time horizons, and evaluation metrics.

[1]: https://www.sciencedirect.com/science/article/pii/S016920702... [2]: https://arxiv.org/abs/2310.13357 [3]: https://mlcontests.com/state-of-competitive-machine-learning...

Re:Moirai: A time series foundation model for universal fore... by vermorel on Tuesday March 26, 2024 @05:52AM (#39825904)
Our basic low-dimensional parametric model landed No1 at the SKU level at the M5, see my lecture https://www.lokad.com/tv/2022/1/5/no1-at-the-sku-level-in-th... (more references at the bottom)
Re:Moirai: A time series foundation model for universal fore... by hcarlens on Tuesday March 26, 2024 @06:06AM (#39825998)
Interesting, thanks for sharing!
Re:Moirai: A time series foundation model for universal fore... by wenc on Tuesday March 26, 2024 @05:39PM (#39833177)
A recent thread on Amazon’s new Chronos forecasting model showed that an ensemble of simple models outperformed it (a highly parametrized transformed model) on the M competition datasets.

https://github.com/Nixtla/nixtla/tree/main/experiments/amazo...