News for hackers. Stuff that matters.
Re:Adversarial image attacks are no joke by aneutron on Monday November 29, 2021 @12:41PM (#29382691)
What you are proposing are what I think would be called a security theater.

It gives the illusion of security, but they would absolutely not deter a determined threat actor.

The only reason that the water supply isn't poisoned is it's unpractical for a single person to conduct the whole exploit chain: Construct the poison in enough quantities, gain access to facilities supplying the water, and actually throwing the compound in it. It's unpractical even for "underground" types. Especially the quantities required.

Mathematics and computer science is a different story in my opinion. You cannot restrict science or thought. You can try, but good luck. The most you can do is delay it. If there is an attack that enables someone to flip a Tesla on the road (as suggested below), the security theater will hide the attack from common folk, but determined actors will reach it eventually, and at that point, they can deploy it as they wish. And in contrast to the water plant, the logistical endeavor to exploit it is absolutely easy in comparison: slap a sticker on your vehicle.

Security by obscurity or by theater is rarely a good strategy in my opinion. We should absolutely be transparent about these kind of things, and allow researchers full access to develop attacks against these systems, and effectively communicate when they are found.

Re:Adversarial image attacks are no joke by tshaddox on Monday November 29, 2021 @04:28PM (#29385521)
It’s pretty easy for a single person to modify or remove an important street sign. Some kids stole traffic signs and were convicted of manslaughter when someone ran a (missing) stop sign and killed another driver. https://www.washingtonpost.com/archive/politics/1997/06/21/3...
Re:Adversarial image attacks are no joke by GhettoComputers on Monday November 29, 2021 @06:44PM (#29386898)
Looks like they were innocent. https://www.law.umich.edu/special/exoneration/Pages/casedeta...

> In 1998, the defense filed a post-conviction motion for a new trial. At a hearing on the motion, several witnesses testified that they had driven by the intersection days before the accident and the stop sign was already down. Some of the witnesses said that after the charges were filed, they reported to both Hillsborough County Sheriff’s detectives and the prosecution that the sign had been down for days, but the information was disregarded. One of the witnesses said she spoke to the prosecutor who disregarded the report and replied that she intended to “burn their ass,” referring to the defendants.

>The motion for a new trial was denied, and the defendants appealed the decision. In March 2001, the Florida Court of Appeals reversed the manslaughter convictions of all three defendants. The court held that the prosecution had made improper comments during closing argument. The court did not reverse the grand theft convictions.

Re:Adversarial image attacks are no joke by tshaddox on Monday November 29, 2021 @06:59PM (#29387016)
Fair enough, I'm not very familiar with the details of that case, but my overall points are that 1) it's easy to remove traffic signs (and thoughtless kids probably do it every so often as a dumb joke), 2) this can definitely lead to bad accidents (with human drivers), and 3) the legal system will likely at least consider whether the people who tampered with the traffic signs are culpable for the accidents.
Re:Adversarial image attacks are no joke by gh0std3v on Monday November 29, 2021 @04:28PM (#29385524)
> What you are proposing are what I think would be called a security theater.

I don't think putting people to prison for, say, flipping a Tesla by screwing with its computer vision algorithm is security theatre. Rather, it's accountability. I'm pretty sure most people are aware that you cannot stop a determined attacker from breaking a system (which is exactly why Spectre mitigations were implemented as soon as the vulnerability was discovered: it's hard to exploit, but still possible).

Defining a legal code for exploiting computer systems through their hardware or their software is not security theatre, it's to ensure that we have a system to punish crime.

Re:Adversarial image attacks are no joke by ineedasername on Monday November 29, 2021 @07:33PM (#29387282)
The theater is in the (somewhat) illusory notion that precautions could prevent it from happening. Prosecuting a crime is absolutely not the same thing as actual security. If a modestly funded department at a university can do this, it's within reach for pretty much any state-level actor. And just like deepfakes are much easier & available for scammers today than they were 5 years ago, the same will go for adversarial images.

5 years ago it would have been pretty much unthinkable that a ransomware attack could actually take down most of the eastern US petrol pipeline infrastructure but here we are, no one prosecuted, and apparently the only thing stopping other high profile attacks is the forebearance and self-policing of the thieves themselves.

Re:Adversarial image attacks are no joke by nerdponx on Monday November 29, 2021 @07:41PM (#29387370)
Making it very expensive to do a thing still reduces the chance of someone doing the thing. How many more murders of passion would happen if murder wasn't illegal?

Laws against murder don't prevent murder from ever happening, but they ensure that committing it is weighed against very high costs.

Perhaps there are other ways to reduce the chance of bad things happening, like reducing opportunities for the bad thing to happen in the first place (eg. not overly relying on computer vision).

Re:Adversarial image attacks are no joke by eximius on Monday November 29, 2021 @12:56PM (#29382876)
> It gives the illusion of security, but they would absolutely not deter a determined threat actor.

Sure. And the threat of jail/imprisonment doesn't deter determined murderer's. It doesn't mean we shouldn't put deterrents.

Re:Adversarial image attacks are no joke by p_j_w on Monday November 29, 2021 @02:06PM (#29383800)
>It doesn't mean we shouldn't put deterrents.

GP doesn't say we shouldn't, but rather that it's not good enough.

Re:Adversarial image attacks are no joke by eximius on Monday November 29, 2021 @03:01PM (#29384493)
Generally calling something security theatre has an implication that it shouldnt be done because of its inefficacy and the availability of robust alternatives (e.g., port knocking is theatre when we can have robust security on known ports with minimal configuration and cryptography).
Re:Adversarial image attacks are no joke by aneutron on Monday November 29, 2021 @03:14PM (#29384664)
While I do agree that security theater does have a connotation for things that have no reason to be done, I only meant that it's not enough. It's theater in the sense that it would only provide a sense of safety, not solve the actual underlying issue or vulnerability class.
Re:Adversarial image attacks are no joke by ghaff on Monday November 29, 2021 @03:36PM (#29384914)
In general, very little is ever enough to completely prevent some sort of determined targeted attack, especially if the attacker doesn't care whether they're caught or not.
Re:Adversarial image attacks are no joke by ineedasername on Monday November 29, 2021 @07:35PM (#29387299)
It's also theater if the ratio between actual protection and perceived protection is highly disproportionate, like with the TSA in the US.
Re:Adversarial image attacks are no joke by tshaddox on Monday November 29, 2021 @04:29PM (#29385534)
Depends what you mean by “not good enough.” It’s obviously not perfect, like all our laws and systems for preventing crimes.
Re:Adversarial image attacks are no joke by rstuart4133 on Tuesday November 30, 2021 @05:03PM (#29398092)
For what it's worth, the threat of jail / imprisonment has no effect on determined murders whatsoever. In fact it has no effect violent crimes in general. From https://www.vera.org/downloads/publications/for-the-record-p... :

> The weak association between higher incarceration rates and lower crime rates applies almost entirely to property crime.16 Research consistently shows that higher incarceration rates are not associated with lower violent crime rates.

It does make sense. If a person is committing a crime in the hopes of material gain, reducing that material gain by imposing a negative gain if they get caught should deter them.

It doesn't seem like a person committing a crime of passion would be using that sort calculus. And it turns out in this case intuition is right: the figures say they don't. Ergo the threat of jail has no effect on the number of murders committed.

Re:Adversarial image attacks are no joke by kingcharles on Monday November 29, 2021 @09:36PM (#29388218)
Absolutely we need deterrents, otherwise chaos.

What deterrents is one of the hardest problems society has ever grappled with. How do we stop antisocial behaviours? Prisons (a modern punishment) do not seem to work, for a multitude of complicated reasons. This is coming from someone who has been through the system.

Re:Adversarial image attacks are no joke by mgraczyk on Monday November 29, 2021 @12:48PM (#29382780)
Right, and the protections for poison are also security theater for the same reason. In the real world that's ok.

> The only reason that the water supply isn't poisoned is it's unpractical for a single person to conduct the whole exploit chain

It's a quantitative question, just like with computer vision. If you don't like the poison example, consider viral DNA, which is also dangerous in the right hands and does not require massive supply chain control. Not everyone has access to a driving dataset like Teslas, and it would be difficult to trick a Tesla without such a dataset.

We should allow researches to develop attacks, just like we should allow researchers to study poisons, DNA, and viruses.

Re:Adversarial image attacks are no joke by flerovium on Monday November 29, 2021 @06:39PM (#29386842)
The lab leak theory isn't necessarily true, but this analogy isn't as conclusive as you make it sound...
Re:Adversarial image attacks are no joke by adrianN on Monday November 29, 2021 @11:31PM (#29388987)
Our whole society is based on the assumption that there are very few determined threat actors. Literally nothing would function if not almost everybody would agree that it's a bad idea to try and break it. I don't think you can change that without making society an unlivable hell.
Re:Adversarial image attacks are no joke by xyzzyz on Monday November 29, 2021 @01:46PM (#29383537)
Construct the poison in enough quantities, gain access to facilities supplying the water, and actually throwing the compound in it.

Gaining access is rather easy. You can easily fly drones over most of reservoirs and dump whatever you want into them. Making strong poisons is also relatively easy, eg. dimethylmercury can be easily synthesized by any chemistry graduate.

Re:Adversarial image attacks are no joke by jcims on Monday November 29, 2021 @02:23PM (#29384006)
You can just pump it back into the municipal water supply from the comfort of your own home (or better yet, someone else’s). You may need to work around a backflow preventer but that’s not too difficult.
Re:Adversarial image attacks are no joke by josefx on Tuesday November 30, 2021 @01:42AM (#29389594)
This results in a dead chemistry graduate.