Works if you have enough domain knowledge to prescribe the functions (priors). Not so good if it's a poorly understood or niche domain.

Sorry for ignorance. Did you mean to say Simple Linear regression or something else? Do you have any reference for that?

Not OP, but as an example, imagine you want to fit the time series of measured monthly CO2 at Mauna Loa:

https://www.esrl.noaa.gov/gmd/ccgg/trends/

Just by looking at the graph, you can see that fitting a linear combination of a constant term, t, t2 and sin(w*t) will give you a very accurate model that has five tuning parameters (four weights and one frequency).

Yes, it's an extension of linear regression. You can incorporate different basis functions to model trend, seasonality, the effects of external regressors, different frequency components, etc. It gives you a lot more control over the forecast model.

Are you referring to Generalized Additive Models (GAM)?

I think GAMs are more for the case where you have many underlying input variables and a single resulting response.

This is a less studied senario for time series research: contextual forecasting. With enough contextual information, forecasting doesn't need to squeeze the information from its own history that hard.

Here's an explanation: https://www.youtube.com/watch?v=rVviNyIR-fI

It's basically fitting on non-linear transformations of x.

You can fit a basis function regression model with least squares.