Re:How the Brain Creates a Timeline of the Past
by jimfleming on Tuesday February 12, 2019 @03:09PM
(#19147030)
Hopefully this leads to better temporal priors for machine learning models. Sequence encoding frequently uses position coding[0] but the typical approaches (e.g. low frequency sin waves) are fairly naive. In reinforcement learning we have (somewhat obvious) works that show that providing explicit time representations help[1]. Simply throwing a recurrent model at a sequence may pick up simple ordering but improving the temporal representation would improve data efficiency.
[0] https://arxiv.org/abs/1706.03762
[1] https://arxiv.org/abs/1712.00378