News for hackers. Stuff that matters.
Re:The first big AI disaster is yet to happen by beloch on Wednesday June 11, 2025 @09:25PM (#44253521)
This is a rare combination of awful factors, but most of them are all too human.

Humans can sift through sigint (Cell phone calls, text messages, network access, etc.) to identify and locate targets. This is labour intensive and prone to error, but it can be done. In the past, such efforts were reserved only for high ranking members of Hamas and, if civilians were killed in the process, the IDF accepted that because of the potential rewards. Note that civilians dying is usually part of the plan in such operations. The "Where's daddy?" tool is designed specifically to identify when targets are in their homes with their families so that they can be bombed along with their families.

Lavender has allowed the IDF to perform the same kind of identification/location for anyone even peripherally involved with Hamas. The IDF has admitted to accepting casualty ratios of 20:1 (civilians to Hamas), and the real ratio is probably higher. If the AI fingers somebody, they're Hamas unless extraordinary evidence proves otherwise. Instead of scrutinizing what comes out of Lavendar, the IDF has shut journalists out of Gaza to reduce the odds of extraordinary evidence ever coming to public light.

The real problem here is not that Lavendar makes mistakes. Humans doing the same job would too. The problem is that the IDF has so completely dehumanized Palestinians that, without questioning their digital oracle, they are willing to drop bombs on large numbers of civilians if there's a chance, according to Lavender, one of them might be involved with Hamas in some way. That's not an AI disaster. That's a human disaster.

Re:The first big AI disaster is yet to happen by ethbr1 on Thursday June 12, 2025 @12:21AM (#44254293)
The IDF is using AI to whitewash ethics in the same way genai is using it to whitewash copyright.

Historically, things have gone badly for society when ills were mostly constrained by potential culpability and prosecution of humans... and then that restriction was suddenly removed by technological advancement.

And the cybersecurity industry / CISOs (aka fall people, giving someone responsibility without authority) are a poor pattern to use to effectively limit tail risk.

At minimum, something more like financial auditing / engineer sign-off should be put in place around productionalized AI, preferably via a third-party.

Re:The first big AI disaster is yet to happen by originalvichy on Thursday June 12, 2025 @02:35AM (#44254801)
This is the chilling part. When Mossad hunted down Eichmann, some of his defense could be summarized as the cliche of ”I was just following orders”. Now with this tooling, they are learning from the past, and centralizing the ”orders” into a non-human machine, which means only a few people (if ever) would stand trial if they were accused of crimes against humanity: minister of defense, some procurement guy, some technical lead. That’s it.

Will our laws of war account for soldiers getting orders from algorithms? Will people get away with killing innocent civilians if a model paints them as a target?

Re:The first big AI disaster is yet to happen by sitkack on Friday June 13, 2025 @03:40AM (#44266543)
We were able to largely ban chemical weapons and so far have prevented the use of nuclear weapons a second time. Maybe we can do this with AI?

Someday there will be a large shrine in Gaza to all the lives that made AI in war a morally reprehensible thing.

Re:The first big AI disaster is yet to happen by ethbr1 on Friday June 13, 2025 @11:33AM (#44269464)
The reason we were able to ban chemical weapons and nuclear weapons, to the extent we did, is that it was in everyone's self-interest.

In contrast, there's a reason major militaries haven't ratified the land mine treaty: they work.

Unfortunately, AI driven weapons are going to effective in terms of decreased response time and increased automomy.

Ergo, the best we can aim for should be enforced guidelines around its use. Especially with regards to human authority/responsibility.

Re:The first big AI disaster is yet to happen by MotiBanana on Thursday June 12, 2025 @08:26AM (#44256946)
> The IDF has admitted to accepting casualty ratios of 20:1 (civilians to Hamas)

No such admission exists, and 20:1 is a made up number that can't be backed by any credible source.

Re:The first big AI disaster is yet to happen by beloch on Thursday June 12, 2025 @03:03PM (#44261740)
Re:The first big AI disaster is yet to happen by therealpygon on Thursday June 12, 2025 @03:23PM (#44262064)
Hate to say it, but for being victims of horrible atrocities, they certainly don’t seem to give much thought to committing them, regardless of the justification.

Every human comes up with reasons to make what they did justified; sides don’t matter in that regard. It takes moral fortitude to do what is right in spite of those excuses and instead of what is easy.

Re:The first big AI disaster is yet to happen by sitkack on Friday June 13, 2025 @03:40AM (#44266546)
stop
Re:The first big AI disaster is yet to happen by therealpygon on Friday June 13, 2025 @07:53AM (#44267697)
Stop what, exactly? Or do your personal feelings obscure your ability to recognize a statement of facts that are unrelated to politics?
Re:The first big AI disaster is yet to happen by balanced2 on Thursday June 12, 2025 @08:01PM (#44264474)
The article is quite confusing for starting with examples of travel accidents which will have a strong human cause but tries to make a case for a AI-only disaster. There is still a strong human factor in every AI tool, for example the best answers come when the model can search for human content. Perhaps humans searching for an AI-only disaster is just a means to shed responsibility from the humans? That seems far more dangerous than anything AI can do.