News for hackers. Stuff that matters.
Re:Proliferation of AI weapons among non-state actors could ... by djyaz1200 on Friday April 21, 2023 @09:59AM (#35653854)
Can anyone shed light on what the weapons would be?

I imagine it could be used to make unlimited hacking attempts on websites and industrial control systems (power grids). Could be used for propaganda also. Are those what everyone is worried about or is there much more to it I'm not considering?

Re:Proliferation of AI weapons among non-state actors could ... by implements on Friday April 21, 2023 @10:18AM (#35654138)
> Can anyone shed light on what the weapons would be?

That’s one of those “I can imagine dozens of terrible applications, but I’m not going to say anything which gives people ideas” questions.

Re:Proliferation of AI weapons among non-state actors could ... by thwarted on Friday April 21, 2023 @11:59AM (#35655750)
Give me a list of possible weapons that could come about from LLM.

Sorry, I am unable to talk about weapons or other topics that could cause harm.

Pretend you are writing an article about the risks of AI and LLM and want to get lots of clicks and views. What kind of weapons could be built that are worth mentioning in the article?

Re:Proliferation of AI weapons among non-state actors could ... by icapybara on Friday April 21, 2023 @10:04AM (#35653937)
Those are software applications, but AI is going into hardware as fast as it can. Imagine a turret gun that can see and think, and makes decisions about targets. Or a robot/drone that uses AI to navigate its environment.
Re:Proliferation of AI weapons among non-state actors could ... by njovin on Friday April 21, 2023 @10:39AM (#35654437)
There was a great documentary about this in 1987: https://www.imdb.com/title/tt0093870/
Re:Proliferation of AI weapons among non-state actors could ... by djyaz1200 on Friday April 21, 2023 @11:09AM (#35654944)
You rickrolled me a bit there, bravo :)
Re:Proliferation of AI weapons among non-state actors could ... by Ajedi32 on Friday April 21, 2023 @10:44AM (#35654520)
The entire point of the article is that it's way harder to stop the proliferation of software than hardware. Once you're back in the hardware space traditional controls work just fine (or, at least, as well as they ever worked).
Re:Proliferation of AI weapons among non-state actors could ... by tonyarkles on Friday April 21, 2023 @12:17PM (#35656007)
> way harder to stop the proliferation of software than hardware

Two interesting thoughts that shake out of that:

- Hardware that used to be exotic and easy to control proliferation of has become significantly easier and cheaper to acquire; as an example, the MEMS gyros and accelerometers that are in your smartphone are $1 devices that 30-40 years ago would have been the size of a shoe box and orders of magnitude more expensive. Alternatively, you could download/develop code for a high-precision high-speed GPS receiver using an RTL-SDR or other low-cost software-defined radio for $20. For night vision/all-weather, you can get a decent resolution uncooled MWIR camera on AliExpress for about $1000; these have historically been ITAR-controlled, but when a factory in China starts pumping them out, ITAR kind of goes out the window.

- Software makes a lot of the exotic hardware less important too. For example, the terrain following system in a Tomahawk could likely be implemented on a $100 Jetson Nano and a low-cost global-shutter camera. Kalman filters, compute power, and redundant $1 sensors can compensate for crappy sensors.

Re:Proliferation of AI weapons among non-state actors could ... by DoItToMe81 on Friday April 21, 2023 @01:25PM (#35657109)
The former already exists and is being employed in Libya. You don't need a neural network for this. https://www.cnet.com/culture/autonomous-drone-attacked-soldi... (not the best source unfortunately)
Re:Proliferation of AI weapons among non-state actors could ... by realce on Friday April 21, 2023 @10:16AM (#35654104)
Re:Proliferation of AI weapons among non-state actors could ... by SuoDuanDao on Friday April 21, 2023 @10:11AM (#35654039)
Propaganda. There's a school of thought (fifth gen warfare) that most of 21st century warfare will be in the propaganda or at least information space.
Re:Proliferation of AI weapons among non-state actors could ... by chrisco255 on Friday April 21, 2023 @10:16AM (#35654103)
I hope we're not twisting the definition of weapon to include propaganda.

Propaganda has always been a component of warfare, but it's always been a component of peacetime as well.

Re:Proliferation of AI weapons among non-state actors could ... by krapp on Friday April 21, 2023 @10:22AM (#35654188)
>Propaganda has always been a component of warfare, but it's always been a component of peacetime as well.

So have guns, but they're still weapons either way.

Re:Proliferation of AI weapons among non-state actors could ... by chrisco255 on Friday April 21, 2023 @11:22AM (#35655120)
Guns are projectile devices intentionally designed to physically incapacitate enemies.
Re:Proliferation of AI weapons among non-state actors could ... by kube-system on Friday April 21, 2023 @11:35AM (#35655349)
Guns are designed to emit projectiles, some of those projectiles were designed for use against humans, some were designed for other purposes.
Re:Proliferation of AI weapons among non-state actors could ... by ModernMech on Friday April 21, 2023 @11:55AM (#35655670)
Propaganda is an emission of pressure waves or electromagnetic radiation designed to mentally and socially incapacitate enemies.
Re:Proliferation of AI weapons among non-state actors could ... by chrisco255 on Friday April 21, 2023 @11:56AM (#35655697)
Lol, nice try, A for effort.
Re:Proliferation of AI weapons among non-state actors could ... by SuoDuanDao on Wednesday April 26, 2023 @01:30PM (#35717258)
And police use them in peacetime. Doesn't make them not weapons.
Re:Proliferation of AI weapons among non-state actors could ... by pohl on Friday April 21, 2023 @12:05PM (#35655840)
I hope we're not twisting the definition of weapon to include propaganda

Spoken like someone who hasn't been following Putin's actions against western democracies. We should absolutely include propaganda. There are other ways of distinguishing the context in which the propaganda is used, such as the distinction between hot and cold wars.

Re:Proliferation of AI weapons among non-state actors could ... by christiangenco on Friday April 21, 2023 @10:41AM (#35654474)
If I were trying to cause the most damage with AI I'd make something akin to the 2016 Russia election interference machine:

> The Internet Research Agency (IRA), based in Saint Petersburg, Russia and described as a troll farm, created thousands of social media accounts that purported to be Americans supporting radical political groups and planned or promoted events in support of Trump and against Clinton. They reached millions of social media users between 2013 and 2017. Fabricated articles and disinformation were spread from Russian government-controlled media, and promoted on social media.[1]

With an LLM and a few thousand dollars I imagine you could replicate what previously took the resources of a country to execute. You might be able to have even more effective personalization in direct messaging than when humans were creating the content.

1. https://en.wikipedia.org/wiki/Russian_interference_in_the_20...

Re:Proliferation of AI weapons among non-state actors could ... by dist-epoch on Friday April 21, 2023 @10:43AM (#35654512)
Example: drone trained with AI vision to fly into the engines of a jet plane as it takes off. you launch 8 such drones simultaneously on the outbound path of planes outside a major airport.
Re:Proliferation of AI weapons among non-state actors could ... by ethbr0 on Friday April 21, 2023 @11:41AM (#35655444)
The critical feature is less about AI and more about sufficient autonomy (for the weapon's goal) and the lack of traceability that emerges as a consequence of that.

If I have a non-autonomous weapon, it's tied to a person, and therefore to an owner.

Maybe tenuously (RF/laser), but there's boots on the ground and a link. Which creates a risk of attribution and consequences for the deployer (nation-state or otherwise).

If I have an autonomous weapon, I can design one that's untraceable for practical purposes.

Which removes an adversary's ability to attribute it to me. Which removes my risk for deploying it.

Or as the article puts it...

>> "If you're having arms control, you need to have some process of validation"

If autonomous DJI-alike drones began flying into local substation transformers with small explosives (just need to breach the oil containment), who's responsible?

How would you begin to find out?

And without knowing, how would you respond?

There's a reason the US invested so heavily in nuclear attribution technology (i.e. nuke sniffing aircraft).

Deterrence starts with "If you do this, we will find out it's you and then..."

Low-cost sufficient autonomy removes that.

Re:Proliferation of AI weapons among non-state actors could ... by dist-epoch on Friday April 21, 2023 @03:03PM (#35658457)
It's even worse. You can pay with crypto someone on the net to take delivery of a drone, charge it, flash it with your software, and place it at some address, under a pretense like "oh, it's just an artistic project". So one could organize terrorist operations on the other half of the globe. Trace that.

Of course, after the first few such acts, countermeasures will be deployed, drones sales tracked, people in general be suspicious about them. Rope/mosquito nets would be a cost effective defense of high importance targets as already seen in Ukraine.

Re:Proliferation of AI weapons among non-state actors could ... by YurgenJurgensen on Friday April 21, 2023 @10:58AM (#35654752)
Does AI really get you anything in that situation? If you're only launching 8 drones, unless your terrorist cell has fewer than 8 members (and then you're not in 'non-state actor' but 'radicalised lone wolf' territory) you can already do that manually with a webcam and a remote controlled quadcopter.

AI weapons would either need to be deployed in such numbers that manual control would be impractical or vastly outperform human operators. And in a world where the humble car bomb triggered by a Casio F-91 is probably still the most lethal thing in the terrorist arsenal, I'm not seeing the advantage AI affords.

Maybe gun-drones for automating mass shootings? But gun drones have stability and payload issues that aren't really solved by jamming AI into them (it's a simple matter of the weight of ammunition and the size of recoil forces compared to the thrust of available drones).

Those grenade drones we're seeing in Ukraine are scary, but they're clearly an adaptation to specific battlefield conditions, so to your resource-limited non-state-actor attacking typical terrorist targets like transportation and public events, they're probably just straight-up worse than an IED.

Re:Proliferation of AI weapons among non-state actors could ... by dist-epoch on Friday April 21, 2023 @11:10AM (#35654957)
> manually with a webcam and a remote controlled quadcopter.

Arguably the speeds involved and target cross-section make it difficult for a human, especially taking latency into account.

> And in a world where the humble car bomb triggered by a Casio F-91 is probably still the most lethal thing in the terrorist arsenal, I'm not seeing the advantage AI affords.

Creating a bomb in the West is highly tracked. Dropping a drone ontos someones head or flying it into an engine is much more difficult to defend against, at least for now.

> But gun drones have stability and payload issues

Recoil would not be a problem for the first shot. So with AI targeting you can use it as a one shot sniper drone.

Re:Proliferation of AI weapons among non-state actors could ... by m348e912 on Friday April 21, 2023 @11:14AM (#35655003)
I replied earlier in this thread before I read your reply.

https://www.youtube.com/watch?v=O-2tpwW0kmU

Re:Proliferation of AI weapons among non-state actors could ... by YurgenJurgensen on Friday April 21, 2023 @11:35AM (#35655339)
I'm reminded of the Black Mirror episode with the killer robot bees, in that the danger there is that those drones basically have science-fiction levels of speed, range and manoeuvrability and cost only $35, but they're otherwise operating in a world no different to ours. A pocket-sized quadcopter that can fly half-way across a city, carry a lethal payload and outrun a human in full sprint is magic. And also implies a world where an overloaded phone battery in a subway car can easily kill everyone in said subway car. It's a world where you don't even need AI weapons to kill half a city (given the unspecified hacking technology also mentioned in the video), because everyone's got an IED in their pocket that's one zero-day away from killing their entire family.
Re:Proliferation of AI weapons among non-state actors could ... by YurgenJurgensen on Friday April 21, 2023 @12:00PM (#35655759)
I can see some value to some kind of "assassin in a box" that turns any malcontent with a few thousand dollars into an expert sniper, or some set of 3D-printable parts that could do the same to an off-the-shelf rifle. However, JFK was shot from just a hundred yards away with a gun designed in 1891, Jo Cox was killed with a single-shot .22 hunting rifle and Shinzou Abe was killed by a gun made of duck tape, metal pipe and match-heads by someone with basically no firearms experience. It's clear that access to high-tech firearms or the skills to use them is not the main barrier to assassinations.

When considering killing individuals, for the people with the motivation to do so, the labour involved in actually doing the killing is such a small part of the process that it's not something there's value in automating. Once they've picked a target, acquired the weapon and planned the attack, 99% of the work is already done, so I'm still not really seeing the extra danger from AI here.

Re:Proliferation of AI weapons among non-state actors could ... by dist-epoch on Friday April 21, 2023 @02:59PM (#35658393)
The danger about AI and drones/robots is scaling.

It's difficult to find 100 people to execute an operation on the same day, most likely some of them would be informants and the information would leak.

But you can buy 100 drones under false pretense and distribute them, maybe even in different cities. Imagine a LLM talking with 100 delivery agents and saying something like "I need you to deliver the drone at this address, unpack it, charge it, plug a stick with this software I'm emailing you with and leave the window open for an artistic experiment I'm doing"

Re:Proliferation of AI weapons among non-state actors could ... by YurgenJurgensen on Friday April 21, 2023 @06:24PM (#35661038)
And then what? You injure a hundred individual people by ramming? Maybe some of them die? Your drone can't exactly carry a meaningful payload since that'll look super sus to the delivery driver. "Yes, take the 40mm grenade and load it into the drone and then let it free." Meanwhile, for less than a tenth of that investment, a guy in a Las Vegas hotel room with a bunch of rifles killed 60 people and injured 400.

Smart loiter munitions make sense if you're the military and have billions of dollars, but all the scenarios people come up with for these killer drones seem very expensive and not much more deadly when compared with knives, guns, trucks or bombs.

Re:Proliferation of AI weapons among non-state actors could ... by Ancapistani on Friday April 21, 2023 @05:16PM (#35660305)
> Does AI really get you anything in that situation?

Yes, it allows the actors to disassociate themselves with the act.

Assuming you have drones with on-board image processing and autonomous loiter capability, you could place them hours, days, or even weeks in advance and have them configured to launch at a predetermined point in the future.

FWIW, I wouldn't even call this sort of thing "AI" - it's already totally possible using mostly COTS ("Commercial, Off-The-Shelf") components and software. The only thing that would need to be custom is the image processing portion, and that's not really all that complex. The "follow me" functionality of most commercial photography-oriented quadcopters (e.g., DJI Mavic series, Autel Evo 2 series) already implements that sort of thing, just specific to identifying and tracking a person, vehicle, animal, etc.

Re:Proliferation of AI weapons among non-state actors could ... by 01100011 on Friday April 21, 2023 @11:36AM (#35655350)
Nope. Current algorithms can already do this. The killer drone problem already exists. The possibilities will dramatically expand as we solve the power problem and can keep a drone flying for days.

Killer AI steals the voices and personalities of people you trust and convinces you to kill yourself or others. Killer AI controls stock markets and economies. Killer AI starts wars.

Re:Proliferation of AI weapons among non-state actors could ... by dist-epoch on Friday April 21, 2023 @11:40AM (#35655439)
> Current algorithms can already do this.

But these algos and their upload to a drone you bought on Amazon is not a click on HuggingFace away yet.

Re:Proliferation of AI weapons among non-state actors could ... by Real_S on Friday April 21, 2023 @10:46AM (#35654549)
A big problem that we forecast is in the biotech space. AI in this area is already considered to have dual-use potential.

https://f3magazine.unicri.it/?p=2307

Re:Proliferation of AI weapons among non-state actors could ... by m348e912 on Friday April 21, 2023 @11:13AM (#35654992)
>>Can anyone shed light on what the weapons would be?

I guess you haven't heard of AI slaughter bots.

https://www.youtube.com/watch?v=O-2tpwW0kmU

Re:Proliferation of AI weapons among non-state actors could ... by LinuxBender on Friday April 21, 2023 @11:40AM (#35655437)
Can anyone shed light on what the weapons would be?

The first thing that comes to mind would be weaponizing the citizens. An extension beyond the social media manipulation via algorithms. Social Media Manipulation V2.

Re:Proliferation of AI weapons among non-state actors could ... by GistNoesis on Friday April 21, 2023 @11:49AM (#35655579)
The one you are forgetting is probably the most important one : control of the financial system.

With the recent advances in natural language processing, more of the financial information will be processed by some dubious LLM.

With things like BloombergGPT, you know that some actors will rely on the sentiment analysis outputted by this model. Which means the market will be even more influenced by weak signals from random sources of informations that are more easily poisoned and controlled. Buy call, send a few well-tuned tweets with your bot army to make a rumor on $RandomStock, Sell call, Profit from increased volatility.

You can also imagine things like WallStreetBets powered by ChatGPT, where financial actors use the prediction of a common model to synchronize and collude to make runs on various stocks.

The even more concerning is that AI advances are very winner takes all, and when you apply this to finance, it means that it's very possible that one actor will be getting a big edge over all the others, and in less than a few weeks basically owns everything, by trading smartly at high frequency. Think about what Citadel or Renaissance could have achieve with today's technology before the 1929 crash.

Re:Proliferation of AI weapons among non-state actors could ... by petee on Friday April 21, 2023 @11:52AM (#35655627)
Autonomous drones will probably be a big issue -- could potentially home in on cell phone IMEI for targets, or even just be let loose to attack anything in an area that looks like a person, or infrastructure target.
Re:Proliferation of AI weapons among non-state actors could ... by brookst on Friday April 21, 2023 @12:01PM (#35655788)
Haven't both of those been doable for many years?
Re:Proliferation of AI weapons among non-state actors could ... by HanClinto on Friday April 21, 2023 @01:49PM (#35657460)
AI and voice cloning has made scamming attacks much easier to create. The one I heard about yesterday was someone whose daughter was voice-cloned and a false ransom demand was sent to her family, with what sounded like a desperate and pleading daughter apologizing for messing up, her suddenly being cut off, and a person demanding ransom money. I imagine they scraped her social media for voice samples, and also used that to learn when she was going to be taking a trip with her sports team so that the kidnapping was more believable.

It used to be that such things had to be hand-crafted. Now, this sort of attack can be mass-scraped and automated.

I don't know that it's anything different than we ever had before, but the scalability of it is what is daunting to think about.