News for hackers. Stuff that matters.
Re:Ask HN: What is the apocolyptic scenario for AI “breaking... by ifyoubuildit on Friday April 14, 2023 @01:15PM (#35571999)
Consider some of our most advanced computer viruses and worms. What if you combine those with an LLM that has been prompted to take over every machine on the planet?

If an LLM figures out (or is trained to, whatever language you prefer) how to distribute itself, how would you ever go about cleaning that up? We're probably at least a few steps away from that currently, but how many?

Re:Ask HN: What is the apocolyptic scenario for AI “breaking... by dragonwriter on Friday April 14, 2023 @01:18PM (#35572036)
> What if you combine those with an LLM that has been prompted to take over every machine on the planet?

Or, without a melodramatically villainous human action in the mix, one which has been prompted to perform some innocuous task, but given inadequate safeguards, and for which it reasons (perhaps accurately, with the paramters it has been given) taking over every other computer will be useful in more efficiently carrying out.

Re:Ask HN: What is the apocolyptic scenario for AI “breaking... by ifyoubuildit on Friday April 14, 2023 @01:20PM (#35572071)
Right, good point, the innocuous case is probably most likely. Either way, once it happens, incredible amounts of damage would be done very quickly.
Re:Ask HN: What is the apocolyptic scenario for AI “breaking... by yeck on Saturday April 15, 2023 @02:08PM (#35583027)
Accidents are a big risk, but don't underestimate the number of individuals and small groups that have fundamentally anti-social philosophies. I think of groups like antinatalists, doomsday religious groups, traditional terrorist orgs. There is even a sort of post-human futurist that sees humanity's purpose being to create a great super intelligence to replace us (may be overlap with this group and antinatalists).

So, I think there we be more chances for legit accidents, but also some motivated people intentionally trying to cause harm.

Re:Ask HN: What is the apocolyptic scenario for AI “breaking... by ActorNightly on Friday April 14, 2023 @02:17PM (#35572826)
>We're probably at least a few steps away from that currently, but how many?

Lets say you are an IT specialist in charge of a data center. Given a single chat box with an LLM on the other end, what would have to occur for you to give it access to the systems?

Because thats what it would take for AI to start taking over computer systems. Computers arent black boxes where some amount of correct input results in an exploit. Its totally possible to have a fully secure system.

Re:Ask HN: What is the apocolyptic scenario for AI “breaking... by ifyoubuildit on Friday April 14, 2023 @02:30PM (#35573036)
> Given a single chat box with an LLM on the other end, what would have to occur for you to give it access to the systems?

Is this supposed to be hard? I setup a process that pipes input/output from the chat agent to a terminal and give it a prompt that contains whatever it needs to login.

That probably doesn't lead to anything too catastrophic happening today, but that's giving the chat box access to the data center.

> Its totally possible to have a fully secure system.

A lot of things are possible. It's a lot easier to have a less than fully secure system.

Re:Ask HN: What is the apocolyptic scenario for AI “breaking... by ActorNightly on Saturday April 15, 2023 @09:08PM (#35586302)
I mean like you are aware that an AI is on the other end, and know you shouldn't give it access.
Re:Ask HN: What is the apocolyptic scenario for AI “breaking... by chatmasta on Saturday April 15, 2023 @05:56PM (#35584957)
> Its totally possible to have a fully secure system.

No, it isn't. As one example, the iPhone has been out nearly 20 years, with hundreds of the best security minds in the world trying to lock it down, and it still gets jailbroken.

Re:Ask HN: What is the apocolyptic scenario for AI “breaking... by ActorNightly on Saturday April 15, 2023 @09:16PM (#35586352)
Sure, if you have physical access to a device, you can pretty much put it under a microscope, solder in your own custom chip in the right places, intercept memory and obtain root access.

This has nothing to do with the conversation of an AI hacking computers remotely through only data comm channels.