News for hackers. Stuff that matters.
Re:Geoffrey Hinton leaves Google and can now speak freely ab... by ActorNightly on Monday May 01, 2023 @04:13PM (#35778491)
... except if ordinary citizens have access to offline versions of advanced AI to use.

Prompt: Give me plans for an anti drone weapon that I can use without being detected.

This is why AI development needs to be accelerated, not put on hold. Companies can and will continue research because that is where the money is. If everyone else is scared of the hypothetical sentient AI boogeyman, then ordinary people will get left in the dark.

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by jehb on Monday May 01, 2023 @04:41PM (#35778849)
I don't see how this scenario justifies AI development being accelerated - could you put some more color on that?

Regulated, sure. Open sourced, probably. But accelerated?

I'm not sure accelerating an arms race is going to help those who are least currently able to take advantage of arms.

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by TylerLives on Monday May 01, 2023 @05:31PM (#35779454)
"Regulated" means that you can't use it but they (the government and companies working for it) can. Whether that's a good or bad thing is debatable, but that's what it means.
Re:Geoffrey Hinton leaves Google and can now speak freely ab... by ActorNightly on Monday May 01, 2023 @09:47PM (#35782177)
Step back into the programming world of 1900s where hacking was way easier and more prevalent then today.

If you were to change the course of history, would you rather have more or less people and open source projects working on things like https, memory safe languages like Rust, 2 factor authentiaction, e.t.c?

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by yboris on Monday May 01, 2023 @04:28PM (#35778688)
Are you excited that an AI could, in the future you describe, spit out correct instructions for creating a more-dangerous virus than COVID to anyone who asks?
Re:Geoffrey Hinton leaves Google and can now speak freely ab... by SirMaster on Monday May 01, 2023 @04:47PM (#35778930)
As long as the AI (that anyone can access) can also spit out an equally powerful antiviral.
Re:Geoffrey Hinton leaves Google and can now speak freely ab... by dmbche on Monday May 01, 2023 @05:12PM (#35779236)
Yeah - that's not how that works I believe. Some problems are harder than others, and the optimal virus it could produce could take orders or magnitude more time/computation.(edit:to produce an effective antiviral)

Also, imagine any one of the billionaires buying all the computing power they can to do something nefarious?

Or the amount of computing power the US could use to produce targetted bioweapons? How could the public compete?

That's without imagining that they could worm(I believe it's been a little bit) most peoples devices and extract some computing power from that.

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by airgapstopgap on Monday May 01, 2023 @06:33PM (#35780175)
That's what you believe but it's not necessarily correct. You assume asymmetry in favor of attacker, but this patently does not apply to e.g. cryptography; the way it's going, we would get more, not less security out of AIs, by automating testing and audits and formal proofs. And, importantly, defense is a common good; best practices could be easily spread, and applied in an economical way with AI, whereas attackers work on their own.

Many functions are asymmetrical in favor of defense. Viruses, too, are not magic; the more sophisticated and powerful its mechanism of action, the longer its code has to be, the worse it is at spreading and surviving the elements (consider how fragile HIV is). Viruses are already tremendously optimized by selection, due to very quickly replication and constant pressure of immunity and medicine. You'd think COVID is merely a warning, but mechanistically it's probably very close to the strongest attack feasible with our biology. Not the most virulent by a long shot; but very good at overcoming our generic defenses.

Crucially it wasn't created with AI. Without any AI, we know perfectly well how to make super-COVIDs, it's limited by accessibility of hardware for microbiological research, not compute or algorithms.

Rapidly designing antivirals, on the other hand, does benefit from AI.

You display a powerful negativity bias which is pervasive in such conversations. You completely ignore AI as a force for good and consider it as, essentially, an offensive capabilty, from which it follows that it must be handed over to incumbents (I take issue with this logic, of course). But that's a self-fulfilling prophecy. Any advantage, centralized enough, becomes an instrument of oppression.

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by dmbche on Monday May 01, 2023 @07:52PM (#35781036)
Could you describe my strong negative bias? I have worries that come to mind - just like people were worried that the atom bomb would burn the atmosphere - and I think they are fair.

I have a hard time understanding your point - not a jab, genuinely- I agree with your last point, where any advantage being centralized becomes an instrument of oppression, and that's mainly where my issue with it lies.

I'm not a doomer at all, I'm personally not afraid of AI. I'm just extending the logic of the previous commenter.

AI could overcomes a lot of problems, for a lot of people. Talking out of my ass, but say Jeff Bezos wants to start a lab to make super-covid or whatnot, and his hurdle is having access to restricted hardware - how hard is it to get the AI to design the hardware?

Regulation of anything becomes basically impossible - and I think that's enough of a worry in itself. (Edit: to clarify, abssence of regulation brings us back to your final point - centralized power leads to oppression. Regulation is supposed to make power less centralized, other than for the common good (yeah yeah I know), so removal or regulation means untethered power for the already powerful.)

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by airgapstopgap on Monday May 01, 2023 @08:55PM (#35781652)
OP:

> As long as the AI (that anyone can access) can also spit out an equally powerful antiviral.

You:

> Yeah - that's not how that works I believe. Some problems are harder than others, and the optimal virus it could produce could take orders or magnitude more time/computation.(edit:to produce an effective antiviral)

With «not how that works» you, I think, implied that there's no reason to expect that proliferation of AI could offset (or indeed decrease) the risk from AI accelerating GoF research. Admittedly I'm not sure specifically about someone's local model designing an antiviral to a new pandemic, that'd certainly happen first on an institutional cluster. But local systems can still assist with e.g. monitoring environment data for new DNA signatures and reporting curious finds.

Anyway, I understood this, in conjunction with other risks you pitched, as a general issue of AI capabilities not offsetting AI risks. I believe this needs better arguments, because many real-world scenarios seem advantageous to the defending side, even when it's "weaker" in terms of resources, and disadvantageous to attacks, which run against natural constraints. An AI filter can see past clever attempts to hide signatures of a spam message (perhaps well enough that passing spam will just stop looking like anything a human would write or read and will be detected by simple heuristics). An AI-fortified firewall will be vastly more reliable than anything we've got now, possibly strong enough to ward off superintelligent attackers. An AI biomed assistant can design vaccines and medicines against entire classes of pathogens, in a way that cannot be overcome just by generating more variants in a wet lab. This is not wishful thinking – it's a very real question. People often fear that AI proliferation as something like everyone getting tabletop nukes, and I think this is an entirely wrong analogy, because it's impossible for physical reasons to build nuclear-powered shields or something; but in the realm of resource-constrained intelligence «that's not how it works».

> and his hurdle is having access to restricted hardware - how hard is it to get the AI to design the hardware?

Pretty hard. But more importantly, everyone interested knows the designs. It's just capital-intensive to the point of impossibility, you need a lot of precision and high-purity materials, so you're forced to buy from established vendors. People tend to overestimate the importance of secrecy in keeping the world livable; I think it's largely a result of propaganda by state security, which is constitutionally biased towards this line of thinking.

In the limit of this logic with AI helping design some precursor to a threat, you'd be just left arguing that AI can make civilization so efficient, any crackpot wannabe comic villain will be able to hide a full supply chain, from mining raw minerals to microchips and bioweapons, on his Texan ranch. Some people bite that bullet, and sure, I think that is doable. But are you sure that a civilization of such logistical prowess would be anything like our one? That it would still be vulnerable to crackpots spreading COVID? That it wouldn't just crank out, say, a few billion UV air purifiers for good measure, because that'd be cheaper than checking?

Be that as it may, I'll pick the prospect of that civilization over the current one, to say nothing of stagnation AI-risk hall monitors want to impose.

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by dmbche on Monday May 01, 2023 @09:14PM (#35781831)
What's GoF research?

However AI can give you an advantage, governements and millionaires will have further access to it. And asymetric advantages are not exclusive to the "good side", as I'm sure you can imagine.

I'm not sure about your thing about nuclear powered shield. What are you talking about?

And about your tangent on the supply chain - I doubt Jeff Bezos has issues getting his hands on anything really- including the materials needed to make one lab? The guy makes rockets, how hard is it to hide enough material for a single building? And you have an AI to ask for help - the only safeguards we've put on as a society is regulation, and this is putting that in jeopardy to my understanding

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by airgapstopgap on Monday May 01, 2023 @11:17PM (#35782891)
Gain of function.

Yes, strong actors will have further access to AI, just as they have to everything else. I believe that on net, scaling properties in this domain are such that proliferation of AI democratizes the world rather than the other way around. The core advantage of strong actors is being able to employ capable (smart) people, after all, and AI diminishes that edge.

> I doubt Jeff Bezos has issues getting his hands on anything really- including the materials needed to make one lab?

Precisely. If he wanted to kill us all with super-Covid, he probably would have pulled it off. Which is my point: it's not the lack of AI that prevents this scenario.

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by dmbche on Monday May 01, 2023 @11:25PM (#35782946)
Thanks for the chat!
Re:Geoffrey Hinton leaves Google and can now speak freely ab... by gregw2 on Monday May 01, 2023 @05:38PM (#35779551)
If you are scientifically minded, I think you should consider how the second law of thermodynamics makes problems for your assumptions.

If you are monotheistically-minded, consider "Satan's ratchet": It's always easier to lie, kill and destroy than to disseminate truth, raise something from the dead, and build.

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by brigandish on Monday May 01, 2023 @08:07PM (#35781206)
And yet we continue to accumulate knowledge at a rate greater than when censorship is greater.

The idea that lies fundamentally outstrip truth is like only telling the first half of the story of the rabbit and the tortoise.

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by Bootvis on Monday May 01, 2023 @05:03PM (#35779104)
What if something bad is much easier to achieve than it’s countermeasure?
Re:Geoffrey Hinton leaves Google and can now speak freely ab... by jamiek88 on Monday May 01, 2023 @05:10PM (#35779204)
And it is. Pretty much always. it easier to destroy than create.
Re:Geoffrey Hinton leaves Google and can now speak freely ab... by smaudet on Monday May 01, 2023 @05:13PM (#35779254)
Who says there is an antiviral for every virus? You can't go doing something because you assume there is a solution to the problem you create - that's irresponsible and if you think that you should be denied all access to modern tech/society.
Re:Geoffrey Hinton leaves Google and can now speak freely ab... by SirMaster on Monday May 01, 2023 @06:04PM (#35779838)
And who says there isn’t?

If this AI is so much smarter than us, who of us is to say it can’t completely solve carbon-based biology or something like that.

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by airgapstopgap on Monday May 01, 2023 @06:38PM (#35780233)
Who says there exists a way out of the regulatory-authoritarian attractor for AI?

Who could've known that nuclear energy is a far lesser threat to humanity than climate change from burning fossils? Certainly not the legions of activists and media producers, who installed the image of green mutagenic goo in people's minds.

Just because you do not even conceive of some risk or don't take it seriously doesn't mean you get to play the Responsible Adult In The Room by pontificating of risks of things people do.

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by rurp on Monday May 01, 2023 @05:07PM (#35779181)
That doesn't work for explosives though. As soon as a bad actor fires off an attack you can't un-blow people up.
Re:Geoffrey Hinton leaves Google and can now speak freely ab... by ActorNightly on Monday May 01, 2023 @10:13PM (#35782428)
People seem to fundamentally misunderstand the problem space of AI.

I assume that you are implying that AI will be able to "figure out" how to synthesize a virus, because something like GPT4 sure as shit not going to be trained on materials on how to specifically synthesize viral weapons.

That "figure out" part is where you make a whole shitload of assumptions, one of which is that P=NP.

Re:Geoffrey Hinton leaves Google and can now speak freely ab... by sangnoir on Monday May 01, 2023 @06:29PM (#35780138)
> Prompt: Give me plans for an anti drone weapon that I can use without being detected.

AI is not magic: such a weapon may not exist and your question would be the equivalent of asking for a unicorn.