Re:AI Is the Black Mirror
by visarga on Sunday December 22, 2024 @03:58AM
(#42485205)
It really does see its past tokens. Attention makes that a core feature, it can introspect on its own past thoughts.
And like humans have to speak one word at a time (serially), so does the model. Distributed activity of neurons funneled through the centralizing bottleneck of serial token prediction. This parallel-to-serial process is the key, I think it's where consciousness is.
And like humans have to speak one word at a time (serially), so does the model. Distributed activity of neurons funneled through the centralizing bottleneck of serial token prediction. This parallel-to-serial process is the key, I think it's where consciousness is.