News for hackers. Stuff that matters.
Re:Joint Statement on AI Safety and Openness by jph00 on Thursday November 02, 2023 @03:09PM (#38118503)
It is not at all true that "AI is no different than proliferating nuclear weapons". The project manager for the Nuclear Information Project said (https://www.vox.com/future-perfect/2023/7/3/23779794/artific...) :

"""

But what we are seeing too often is a calorie-free media panic where prominent individuals — including scientists and experts we deeply admire — keep showing up in our push alerts because they vaguely liken AI to nuclear weapons or the future risk from misaligned AI to pandemics. Even if their concerns are accurate in the medium to long term, getting addicted to the news cycle in the service of prudent risk management gets counterproductive very quickly.

## AI and nuclear weapons are not the same

From ChatGPT to the proliferation of increasingly realistic AI-generated images, there’s little doubt that machine learning is progressing rapidly. Yet there’s often a striking lack of understanding about what exactly is happening. This curious blend of keen interest and vague comprehension has fueled a torrent of chattering-class clickbait, teeming with muddled analogies. Take, for instance, the pervasive comparison likening AI to nuclear weapons — a trope that continues to sweep through media outlets and congressional chambers alike.

While AI and nuclear weapons are both capable of ushering in consequential change, they remain fundamentally distinct. Nuclear weapons are a specific class of technology developed for destruction on a massive scale, and — despite some ill-fated and short-lived Cold War attempts to use nuclear weapons for peaceful construction — they have no utility other than causing (or threatening to cause) destruction. Moreover, any potential use of nuclear weapons lies entirely in the hands of nation-states. In contrast, AI covers a vast field ranging from social media algorithms to national security to advanced medical diagnostics. It can be employed by both governments and private citizens with relative ease.

"""

Let's stop contributing to this "calorie-free media panic" with such specious analogies.

Re:Joint Statement on AI Safety and Openness by chakalakasp on Thursday November 02, 2023 @03:32PM (#38118796)
If you talk to the people on the bleeding edge of AI research instead of nuke-heads (I tend to be kinda deep into both communities), you'll get a better picture that, yeah, a lot of people who work on AI really do think that AI is like nukes in the scale of force multiplication it will be capable of in the near to medium future, and may well vastly exceed nukes in this regard. Even in the "good" scenarios you're looking at a future where people with relatively small resources will have access to information that would create disruptive offensive capabilities, be it biological or technological or whatever. In worse scenarios, people aren't even in the picture any more, the AIs are just working with or fighting each other and we are in the way.
Re:Joint Statement on AI Safety and Openness by tsurba on Thursday November 02, 2023 @04:25PM (#38119596)
I’m pretty sure what communities you are in are not actual research but some hype alarmist bullshit communities, since as a ML researcher absolutely zero of my peers think the things you say.
Re:Joint Statement on AI Safety and Openness by chakalakasp on Thursday November 02, 2023 @07:42PM (#38122001)
You have no clue what communities I’m in other than it self affirms your worldview to assume that they must be irrelevant. To be fair, I’m not telling you anything about myself, so you’re not really needing to take my word for it. And I don’t care enough about you to explain in detail.

Though with a tiny bit of Googling you’ll be able to find several Turing Award winners who are saying exactly what I’m saying. In public. Loudly.

Re:Joint Statement on AI Safety and Openness by jay_kyburz on Thursday November 02, 2023 @03:37PM (#38118877)
Furthermore, there is little or no defense against a full scale nuclear attack, but a benevolent AI should be sufficient defense against a hostile AI.

I think the true fear is that in an AI age, humans are not "useful" and the market and economy will look very different. With AI growing our food, clothing us, building us houses, and entertaining us, humans don't really have anything to do all day.

Re:Joint Statement on AI Safety and Openness by chakalakasp on Thursday November 02, 2023 @03:59PM (#38119185)
In what other sphere of tech does that apply?

"Hackers aren't a problem because we have cybersecurity engineers". And yet somehow entire enterprises and governments are occasionally taken down.

What prevents issues in redteam/blueteam is having teams invested in the survivability of the people their organization is working for. That breaks down a bit when all it takes is one biomedical researcher whose wife just left him to have an AI help him craft a society ending infectious agent. Force multipliers are somewhat tempered when put in the hands of governments but not so much with individuals. Which is why people in some countries are allowed to have firearms and in some countries are not, but in no countries are individuals allowed to legally possess or manufacture WMDs. Because if everyone can have equal and easy access to WMDs, advanced civilization ends.

Re:Joint Statement on AI Safety and Openness by mattnewton on Thursday November 02, 2023 @04:25PM (#38119595)
I mean, hackers aren’t such a problem that we ban developing new internet apps to only licensed professionals by executive order, so that kinda proves the parent posters point?
Re:Joint Statement on AI Safety and Openness by chakalakasp on Thursday November 02, 2023 @07:26PM (#38121877)
The difference is the scale at which a hacker can cause damage. A hacker can ruin a lot of stuff but is unlikely to kill a billion or two people if he succeeds at a hack.

With superintelligent AI you likely have to have every developed use and every end user get it right, air tight, every time, forever.

Re:Joint Statement on AI Safety and Openness by jay_kyburz on Thursday November 02, 2023 @04:24PM (#38119583)
Yes, but the AI that is watching what every Biomedical researcher is doing will alert the AI counselors and support robots and they will take the flawed human into care. Perhaps pair them up with a nice new wife.

Can you imagine how much harder it would be to protect against hackers if the only cybersecurity engineers were employed by the government.

Re:Joint Statement on AI Safety and Openness by hackinthebochs on Thursday November 02, 2023 @04:42PM (#38119852)
>but a benevolent AI should be sufficient defense against a hostile AI.

What on earth could this possibly mean in practice? Two elephants fighting is not at all good for the mice below.

Re:Joint Statement on AI Safety and Openness by jay_kyburz on Thursday November 02, 2023 @06:34PM (#38121296)
Better than one Elephant who hates mice.
Re:Joint Statement on AI Safety and Openness by coryfklein on Thursday November 02, 2023 @06:13PM (#38121024)
And the best path to a benevolent AI is to do what? The difficulty here is that making an AGI benevolent is harder than making an AGI with unpredictable moral values.

Do we have reason to believe that giving the ingredients of AGI out to the general public accelerates safety research faster than capabilities research?

Re:Joint Statement on AI Safety and Openness by coryfklein on Thursday November 02, 2023 @06:14PM (#38121060)
Color me surprised that that the project manager for the Nuclear Information Project is in fact a subject matter expert for nuclear power and not AGI x-risk. Why would they be working on nuclear information if they didn't think it the most important thing?