News for hackers. Stuff that matters.
Re:Early rogue AI agent activity and attempts to hack found ... by ambicapter on Thursday September 24, 2026 @06:03PM (#49837309)
You wouldn't be suspicious if the nuclear engineers kept on feeding their reactor and building more powerful reactors while they went crying to the news?
Re:Early rogue AI agent activity and attempts to hack found ... by jagraff on Thursday September 24, 2026 @07:13PM (#49838059)
In the nuclear case, no, because I’d assume they still have a mortgage to pay.

In the AI case, no, because I think the engineers believe that there is a high probability of enormous upside as well, if it doesn’t kill us all.

Re:Early rogue AI agent activity and attempts to hack found ... by bagacrap on Thursday September 24, 2026 @11:40PM (#49839957)
upside for whom?
Re:Early rogue AI agent activity and attempts to hack found ... by jagraff on Friday September 25, 2026 @08:26AM (#49843677)
Depending on who you ask, either for all of humanity (because they will allow us to do things like cure most cancers, develop useful technology, etc) or for the people building them (because they will make them rich enough to buy galaxies) or both
Re:Early rogue AI agent activity and attempts to hack found ... by bagacrap on Saturday September 26, 2026 @12:19AM (#49853169)
The original issue was whether these people are trustworthy. Someone whose main motivation is self enrichment is trustworthy in your eyes?
Re:Early rogue AI agent activity and attempts to hack found ... by jagraff on Saturday September 26, 2026 @08:56AM (#49856061)
I’m not saying they are necessarily trustworthy - I’m just saying that we ought to take their claims seriously and not dismiss them out of hand because they have a financial interest.

I’ll admit I also just don’t understand the idea that someone saying their product is dangerous and could kill all of humanity is doing marketing - I just really don’t understand at all how that could be a marketing strategy. So I tend to believe they are saying it is dangerous because they think it’s true. But maybe I’m just naive.

Re:Early rogue AI agent activity and attempts to hack found ... by bagacrap on Saturday September 26, 2026 @09:51AM (#49856554)
It's telling investors "this is so powerful we could take over the world, better give us your capital."

It's telling governments, "this is so powerful it could cause grave harm to the consumer, please regulate so we can focus on extracting value from the economy instead of this expensive arms race."

I warrant it is quite possible that some in this space are worried they will overshoot the mark. The perfect outcome for them, of course, being fabulous wealth and a society that is otherwise not too much different from today's. What a shame it would be to be super rich and dead!

It does kind of feel like they are trying to ransom the future of humanity. These rogue AI stories are like getting a dismembered finger in the mail.

Re:Early rogue AI agent activity and attempts to hack found ... by ambicapter on Friday September 25, 2026 @09:20AM (#49844270)
Hold up, you think its understandable to keep feeding a melting down nuclear reactor if you have a mortgage to pay?
Re:Early rogue AI agent activity and attempts to hack found ... by jagraff on Friday September 25, 2026 @10:06AM (#49844870)
I think it is understandable to keep doing a job that you think is dangerous or bad for the world because you need money, yes.
Re:Early rogue AI agent activity and attempts to hack found ... by ambicapter on Friday September 25, 2026 @11:48AM (#49846211)
And is there a point where "bad for the world" = "apocalypse scenario" where you think it's no longer acceptable, or getting a salary is always justification for doing any level of harm?
Re:Early rogue AI agent activity and attempts to hack found ... by jagraff on Friday September 25, 2026 @02:07PM (#49847984)
Well if we're talking about AI risk, the justification that I've seen from researchers is that while they do believe that there is a risk of apocalypse scenarios, they believe that they are better positioned than anyone else to prevent that, and also they believe that the upside of AI is so great for the world that it is worth the risk.

Also I'm not describing what I personally think is an acceptable job; just that I understand that sometimes people do jobs that they think are wrong because they need money