News for hackers. Stuff that matters.
Re:AI Is the Black Mirror by visarga on Sunday December 22, 2024 @03:58AM (#42485205)
It really does see its past tokens. Attention makes that a core feature, it can introspect on its own past thoughts.

And like humans have to speak one word at a time (serially), so does the model. Distributed activity of neurons funneled through the centralizing bottleneck of serial token prediction. This parallel-to-serial process is the key, I think it's where consciousness is.

Re:AI Is the Black Mirror by grey-area on Thursday December 26, 2024 @02:29AM (#42513671)
For certain values of introspect.