Personal opinion, but any system that does not have survival instinct is not intelligent in my book. There are many dumb animals in nature who we do not deem intelligent but they still have decent survival instinct. AI systems are not remotely close.

spot on

True. There's a theory that says that intelligence evolved as a tool to outsmart your pray. So top level predators will always have to be smarter than their pray. Therefore, survival needs intelligence.

Personally, I think we are going to hit a hard stop on our current path to AGI. We are looking at intelligence from the top level hoping to somehow duplicate it. What we are getting is a dumb mimic with zero intelligence. We need to start at the bottom and build our way up. So, pick a low intelligence animal and figure out how they perceive the world and survive in the world. Trying to mimic intelligence won't get us to AGI.

Good points. Some people claim the current generation of LLMs have "intelligence" but in reality it is basically a habit- that let's them predict words.

We can try to "teach" an AI system survival instinct, but then it'll most likely iterate/evolve into something that wipes us out. I just don't see them living in symbiosis with us.

> Without an innate understanding of the world, AI can’t predict. And without prediction, it can’t plan.

bingo