Re:A decoder-only foundation model for time-series forecasting
by remontoire on Friday February 02, 2024 @10:50PM
(#39237388)
Probably drawing an analogy to how causal pretrained models go through stages of understanding language, words -> grammar -> meaning. Gwen mentions this experience when training character level RNNs. https://gwern.net/scaling-hypothesis#why-does-pretraining-wo...
[ Parent
| on HN ]
Re:A decoder-only foundation model for time-series forecasting
by horacemorace on Friday February 02, 2024 @11:09PM
(#39237471)
Totally. Also basic temporal scale or cyclic properties. It’s kind of mind blowing that the shape of most recorded human patterns is reducible in this way.