Humans can sift through sigint (Cell phone calls, text messages, network access, etc.) to identify and locate targets. This is labour intensive and prone to error, but it can be done. In the past, such efforts were reserved only for high ranking members of Hamas and, if civilians were killed in the process, the IDF accepted that because of the potential rewards. Note that civilians dying is usually part of the plan in such operations. The "Where's daddy?" tool is designed specifically to identify when targets are in their homes with their families so that they can be bombed along with their families.
Lavender has allowed the IDF to perform the same kind of identification/location for anyone even peripherally involved with Hamas. The IDF has admitted to accepting casualty ratios of 20:1 (civilians to Hamas), and the real ratio is probably higher. If the AI fingers somebody, they're Hamas unless extraordinary evidence proves otherwise. Instead of scrutinizing what comes out of Lavendar, the IDF has shut journalists out of Gaza to reduce the odds of extraordinary evidence ever coming to public light.
The real problem here is not that Lavendar makes mistakes. Humans doing the same job would too. The problem is that the IDF has so completely dehumanized Palestinians that, without questioning their digital oracle, they are willing to drop bombs on large numbers of civilians if there's a chance, according to Lavender, one of them might be involved with Hamas in some way. That's not an AI disaster. That's a human disaster.
Historically, things have gone badly for society when ills were mostly constrained by potential culpability and prosecution of humans... and then that restriction was suddenly removed by technological advancement.
And the cybersecurity industry / CISOs (aka fall people, giving someone responsibility without authority) are a poor pattern to use to effectively limit tail risk.
At minimum, something more like financial auditing / engineer sign-off should be put in place around productionalized AI, preferably via a third-party.
Will our laws of war account for soldiers getting orders from algorithms? Will people get away with killing innocent civilians if a model paints them as a target?
Someday there will be a large shrine in Gaza to all the lives that made AI in war a morally reprehensible thing.
In contrast, there's a reason major militaries haven't ratified the land mine treaty: they work.
Unfortunately, AI driven weapons are going to effective in terms of decreased response time and increased automomy.
Ergo, the best we can aim for should be enforced guidelines around its use. Especially with regards to human authority/responsibility.
No such admission exists, and 20:1 is a made up number that can't be backed by any credible source.
Every human comes up with reasons to make what they did justified; sides don’t matter in that regard. It takes moral fortitude to do what is right in spite of those excuses and instead of what is easy.
https://www.972mag.com/lavender-ai-israeli-army-gaza/
One source stated that human personnel often served only as a “rubber stamp” for the machine’s decisions, adding that, normally, they would personally devote only about “20 seconds” to each target before authorizing a bombing — just to make sure the Lavender-marked target is male. This was despite knowing that the system makes what are regarded as “errors” in approximately 10 percent of cases, and is known to occasionally mark individuals who have merely a loose connection to militant groups, or no connection at all.