News for hackers. Stuff that matters.

A comment on Markov Chains Are the Original Language Models. Parent comment

Re:Markov Chains Are the Original Language Models by PheonixPharts on Thursday February 01, 2024 @11:02AM (#39217346)
LLMs are Markov chains in latent space, it's the latent representation that give them their power, but ultimately there's not as much difference as one would suspect.
[ Parent | on HN ]
Re:Markov Chains Are the Original Language Models by esafak on Thursday February 01, 2024 @05:19PM (#39222240)
They're different because Markov models are stateless whereas LLMs are stateful.

https://en.wikipedia.org/wiki/Markov_property

[ Parent | on HN ]
Re:Markov Chains Are the Original Language Models by Jensson on Friday February 02, 2024 @02:16AM (#39225996)
Current LLMs are stateless as far as we know, their state when computing a new token is only the preceding text tokens, they don't store any metadata or save state from the previous calculations.
[ Parent | on HN ]