Re:The Darwin Gödel Machine: AI that improves itself by rewr...
by telotortium on Friday May 30, 2025 @02:31PM
(#44138811)
What do you even mean by self-awareness? Presumably you don’t mean fish contemplate their existence in the manner of Descartes. But almost all motile animals, and some non-animals, will move away from a noxious stimulus.
Re:The Darwin Gödel Machine: AI that improves itself by rewr...
by littlestymaar on Friday May 30, 2025 @02:56PM
(#44138977)
The definition is indeed a bit a tricky question, but there's a clear difference between the reflex of protecting oneself from danger or pain and higher level behavior that show that the subject realizes its own existence (the mirror test is the most famous instance of such an effect, but it's far from the only one, and doesn't only apply to the sense of sight).
There's no reason to expect self awareness to emerge from stacking enough Lego blocks together, and it's no different for GPT-based neural nets.
In nature, self awareness gives a strong evolutionary advantage (as it increases self-preservation) and it has been independently invented multiple times in different species (we have seen it manifest in some species of fishes for instance, in addition to mammals and birds). Backpropagation-based training of a next-token predictor doesn't give the same kind of evolutionary advantage for self-awareness, so unless researchers try explicitly to make it happen, there's no reason to believe it will emerge spontaneously.