For some kinds of forecasting it doesn't need to be exactly accurate - eg say you are forecasting traffic for how to scale a load balancer, it can be a bit off it will just cost you a little and you can adjust after - a lot of feedback systems dont need full accuracy. You wouldnt use this to forecast whether to buy shares. It basically replaces various traditional curve fitting tools. If you need a really good forecast yes you are going to build a domain model. At least this is my understanding...

> say you are forecasting traffic for how to scale a load balancer, it can be a bit off

Problem is you really don't even need ML for the curve fitting tools. Just ARIMA or Holt Winters is enough and likely faster. Heck, Kubernetes HPA doesn't even implement full PID, just P.

It seems like the main challenge is going to be finding domains where accuracy is important enough to spend inference token money on, but not so important that you'd hire a team to build a custom model. Or maybe that team borrows Timestral weights to fine tune to the domain, but they probably already have a different model customized to the problem?

No affiliation with mistral, but you don't have to purchase tokens for inference. It runs GPUless and is open source, so you're just paying the normal compute cost.

Interesting, I was under the impression that models generally run cheaper and faster on GPUs, even though the hourly rates are higher.

so If I understand correctly classical ML would beat this generic approach, classical ML being feature engineering/selection and curve fitting? My next question would be that wouldn't it be smarter to take an auto research agent or even just a classic grid search of sorts to find those features and best curve to fit your problem using backtesting to evaluate and iterate.

I'm doing time series forecasting for a biological system. I haven't used these foundation models but my experience is that knowledge of how things are related is baked into what variables you put into the model (which can be more than raw data, you can make any derived product you think would be useful) but it's up to the model to figure out how they are related (if I knew exactly how they were related I wouldn't be training a forecasting model).

I'm sure someone who actually knows this stuff has a better easy to view it but my impression was that these time series foundation models aren't trained for a specific dataset but more they are trained on a procedure for how to look at datasets and make predictions. It doesn't seem to lump things into a bucket much more than any other model type (which also are just consistent procedures to look at datasets). It's not like you have radically different model types across different fields.

Forecaster of the Confounding Company here :)

You're right but, if the time series looks like music then you can apply your music domain knowledge. You don't have to only apply knowledge that's general across every domain, part of the magic of large neural nets is that you can do recognition and prediction at the same time thereby using (statistically) things known from training about how to identify a sequence and then also how to continue it.

Sure adding outside info helps even more, but with enough data it should become unnecessary (or so goes the theory).

But why would you throw away that data at all? These are tiny models compared to domain-specific audio/music/image models, and any capacity spent on recognition is capacity that could have been used for prediction.

Is there a reason you don't throw in something like cross-attention on an embedding of the sequence description/data labels? I've trained numerous generative models for audio and images and it's always the case that the unconditional model performs worse, trains slower, etc. than the conditional one (with the best performance from a hybrid using classifier-free guidance or similar scheme).

Again I don't have experience with the domain of time-series forecasting directly, but I would think there are generalities between autoregressive generative models and time-series forecasting, given that both involve predicting the next sample(s) given N regularly sampled past inputs.

>"Is there a reason you don't throw in something like cross-attention on an embedding of the sequence description/data labels?"

Won't you need to attribute a weight to the data coming from the model and the weight for your data?

Do you actually find that for some use cases foundation models work better than AutoARIMA or etc? I do a lot of ecom projections and it’s never been the case for me, most of my cases are best with seasonal naive.

Would love to know actual time series foundation model use cases!

You’re not going to get venture capital funding by using auto.arima

Exactly this. It's a zero-shot model that can understand what a time series is with very little a-priori information. I would assume they've trained across a wide diversity of real world signals.

I interviewed for a company which used time series modelling to predict engine failure on ships. Though they didn't call it that, they called it just "AI" and weren't familiar with the time series literature at all. (Neither was I, but I did at least know that the time series literature was a thing.) They just used a bunch of sensors attached to the engine.

That seems like a scenario where there wouldn't be external sources of information capable of improving your predictions. It also seemed to me like a scenario where good old fashioned statistical methods would outperform LSTMs, which was what they used.

(I didn't get the job - I was offered it, but they rescinded their offer at the last moment, supposedly because their investors had insisted they should hire a web developer instead.)

I appreciate this analysis, but I wonder, setting aside unpredictable current events for the moment, aren’t seasonal and scheduled events also covariates? eg forecasting baseball scores year round?

I can see another use for time-series generative models; to create the right shape of noisy data.

To improve the robustness of machine learning models, artificial noise is often introduced into actual training data. Something like Timestral could simplify the process of generating it.

Forecasting is just trying to identify recurring and long-time trends in data, and you're absolutely right that with a narrow observation window it is more difficult to do so, however if you have enough historical data (i.e. several years), these models are remarkably good at picking up on patterns leading up to e.g. bank holidays, the information is contained in the data.

That being said, more information is of course better, assuming it doesn't introduce bias, like focusing on specific events like holidays too much, when the information is already contained in the historical data, without having to make assumptions about what an upcoming holiday means.

The biggest challenge last time I worked with time series forecasting (which is ~5 years ago) was learning long-term shifts in the baseline of the data (e.g., the general use of electricity always goes up, a lot more solar panels and EVs influence the baseline, etc.).

I can only assume (hope) that there has been some substantial progress in that area though.

(Edited the formatting for a nicer reading experience)

100% agree. Time series forecasting is really bad and there are some theorems that say it is *impossible for large class of series to be forecasted by more than a small number of steps..turning into a straight-line, meanwhile if you have say a handful of important day indicators of the same signal that are proxies, you can recover pretty much exact curve. Better to spend compute in my opinion or variable learning approximate Xt -> yt mapping than the X_t-4..n -> X_t mapping...