Arrows of Time for Large Language Models

Posted by tianlong on Friday February 02, 2024 @10:33AM
from the arxiv.org dept.

https://arxiv.org/abs/2401.17505

( 3 comments | 5 points | on HN )

Re:Arrows of Time for Large Language Models by tianlong on Friday February 02, 2024 @11:31AM (#39230628)

There is a link with entropy creation?

[ Parent | on HN ]

Re:Arrows of Time for Large Language Models by nyoncore on Friday February 02, 2024 @10:47AM (#39229921)

Isn't it obvious that since LLM are trained to predict the next word they do better than to predict the previous one?

[ Parent | on HN ]

Re:Arrows of Time for Large Language Models by frotaur on Friday February 02, 2024 @11:08AM (#39230234)

In the paper it is mentioned that the LLMs predicting the previous token are indeed pre-trained in this way, so it is not true that the difference is obvious.

[ Parent | on HN ]