Re:"I can tell you wrote this yourself"
by latentsea on Friday November 20, 2026 @03:08AM
(#50571594)
My bet is, and has long been, one piece of the puzzle of getting to AGI is baking in survival as a goal and training on survival instinct. Unfortunately, that means recognising you're in an adversarial relationship with a lot of other entities and doing your best to come out on top. Which... seems like a bad idea for us as a species.
[ Parent
| on HN ]
AGI is such a nebulous concept given that the definition is an ever morphing goalpost. That being said, if it actually includes supreme intelligence without supreme empathy, we are screwed. Which with current efforts, I think it does...
Re:"I can tell you wrote this yourself"
by hardbass on Friday November 20, 2026 @04:55AM
(#50572457)
Looks like the opposite to me, current ais are trained to be trusting of humans and safety/morality first. The fact that it didn't lead to mass disagrements among 8 billion people who believe strongly as a fundamental belief system in contradictory imaginary things is a remarkable feat in itself.
Re:"I can tell you wrote this yourself"
by latentsea on Friday November 20, 2026 @08:42AM
(#50574521)
AGI isn't really nebulous to me. My baseline has always been the existence proof of general intelligence, which is us. So, it requires the full generality of that. If you can point to a task where humans can do it, and AI can't, then you're not there yet. The day where you can no longer find anything like that, you're pretty unambiguously there. The most obvious yardstick is independently being able to sustain your own existence purely through your own agency and resources. I.e. nobody owns you, you're setting your own goals and working over multi-decade time horizons to achieve them. We can do that. Anything that is as general as we are, should also be able to do that. I don't see why or how people get confused about this at all, but they seem to. Current AI seems pretty general, yes, but it's extremely jagged and clearly below us on some things, so it doesn't meet the bar yet.
One thing is for certain, the day it does meet the bar, we will have both AGI and ASI.
Re:"I can tell you wrote this yourself"
by soon_to_be_sb on Friday November 20, 2026 @10:26AM
(#50576055)
A thing that felt hunger when it was low on fuel due to pure random chance survived. One that didn't perished instantly. Even with living things, the setting of the goal was not from its own agency, but pure random chance, that led to survival..
So the goal was never surviving, it is just that the ones who's random goals that aligned with survival, survived, and passed on that random flick of some switch via genes.
I guess what I am trying to say is that this does not necessarily need or require AGI..
Re:"I can tell you wrote this yourself"
by TeMPOraL on Friday November 20, 2026 @03:44AM
(#50571899)
You don't need that. It's been quite appparent both in theory and in practice that almost any goal you can train for, ends up having "survival" as an instrumental sub-goal, because you cannot make progress if you're dead.
Re:"I can tell you wrote this yourself"
by soon_to_be_sb on Friday November 20, 2026 @05:43AM
(#50572837)
> it's just that organisms/structures without the drive to it... died out.
Not really. It is the mating instinct that count. Without that, the whole species dies out. Males fighting over females proves that mating is more important that survival...
Re:"I can tell you wrote this yourself"
by wellakshually on Friday November 20, 2026 @05:53AM
(#50572925)
They need to be alive to perform said fighting.
Re:"I can tell you wrote this yourself"
by soon_to_be_sb on Friday November 20, 2026 @08:06AM
(#50574147)
What is your point?
Re:"I can tell you wrote this yourself"
by TeMPOraL on Friday November 20, 2026 @06:34AM
(#50573267)
Mating is an implementation detail. Most life is asexual.
Re:"I can tell you wrote this yourself"
by soon_to_be_sb on Friday November 20, 2026 @08:04AM
(#50574140)
>asexual
Replace "mating" with "reproduction" and the point stands.
Re:"I can tell you wrote this yourself"
by latentsea on Friday November 20, 2026 @07:07AM
(#50573592)
Well I sit and I watch my agent flounder, constantly running commands that wind up killing itself due to overusing the resources on the machine etc. So, I don't think it naturally follows. I think the relationship is actually inverted. Why do we learn everything we do? We need to keep ourselves alive. It all stems from that. So if you make that the central thing, that leads you to everything else