The fear is that instead of an algorithm and a human double checking each other and refining each other that they will simply add. So we end up with the worst decisions of both human and computer.

I hadn't considered that angle before, there needs to be a lot of oversight (in general when we're killing people, not just computers) but I think you're right.

It's adorable how you think it will improve how war is fought. These autonomous systems will always have a manual override and I will never believe that the review process will be given proper oversight. The armed forces will always err on the side of destruction in any potentially vaguely violent scenario. Fear and War is too prosperous to be peaceful.

If this is true, one wonders why they waste all that time writing rules of engagement, and prosecuting those who don't follow them.

We generally only prosecute those who break those "rules of engagement" when they are not ourselves.

This cannot ever be a good thing; drones justify terrorism because it's the only available response to fighters from the "other side".

Thinking we can wage wars without risk to ourselves or our "soldiers" is preposterous.

"Just" wars are a ridiculous idea rooted in superstition and hubris.

    > drones justify terrorism because it's the only available
    > response to fighters from the "other side".

We had similar terrorist attacks before we had drones, read what the likes of Bin Laden said about why 9/11 happened. It's because the infidels are on their homeland, not because they're using some specific technology to attack them.

More generally people resort to terrorism in the face of an overwhelming enemy. The west could relatively easily win the wars they're fighting now in the middle-east with their WWI armies if they were willing to care less about civilian casualties, that's how lopsided the odds are.

    > Thinking we can wage wars without risk to ourselves or our
    > "soldiers" is preposterous.

Drones as a technology would happen regardless of whether or not the armies developing them were concerned about their own soldiers. They're simply the logical next step in technology, e.g. fighter jet development has stalled to some extent not because the technology has reached its limits, but because if you pull more G's than you do now the pilots might die.

1. The current algorithm might be an unpredictable twenty your old in uniform, but there is accountability for their actions: it is somebody's actions. Who is responsible for the illegal actions of AI? You start to have to untangle racially motivated assault against "it shouldn't have done that".

2. People will not come up with "responsible" rules of engagement as you seem to suggest. isBrownPerson() will be coded into the algorithm but not in a discernible way. Sure, you could verify this against a video collection: like we all know the statistics of profiling and incarceration rates for different races.