They sort of address this -

"To be actually a useful tool to military commanders, you want an AI which [...] gives them a reasoned opinion on which of the various options they are considering is most advisable."

With that (far more sophisticated) AI system, leadership could just de-prioritize options with excessive risk of civilian casualty.

The other day I was looking at this – http://www.datalex.org/ [0] – presented with a situation, it asks you questions about it, and then generates a legal opinion on your situation. It shows all its reasoning (in natural language) to arrive at that legal opinion. That's the same basic thing a human advisor does – if you get a formal legal opinion from a lawyer, it doesn't just give you the conclusion, it also explains the arguments and reasoning and considerations it used to arrive at that conclusion.

So I don't see why a strategic military advice platform couldn't do the same basic thing.

[0] Here's a working example: http://beta.datalex.org/app/?rulebase=http://austlii.communi...

It sound more like a prolog program. It takes a set of rules and facts and can infer other facts from there and show you all the steps of the process. Although it is a branch of AI it is not what one thinks of nowdays hearing the term AI. One thinks of some fancy powerful self learning black box, which can do miracles but can't explain how it does it.

I'll rephrase what I'm saying.

What if the AI lies?

What if the AI lies and civilians die because of it?

I think we're pretty far away from the kind of AI that could intentionally trick its operators to achieve a goal.

Giving a reasoned opinion is a huge unsolved problem in AI.

Yes, and I referred to that hypothetical AI as "far more sophisticated".

>What if the AI decides that the course of action that leads to the desired outcome of winning the conflict is to kill civilians

I always figured that was what happened in the film Colossus: The Forbin Project, the computer(s) found a way to ensure / prevent a nuclear war / peace, by taking control of the nuclear arsenals on their own and ensuring peace via their own threat of nuclear weapons.

> the desired outcome of winning the conflict

Requires political victory. War is extension of politics, not alien invasion defense.

Thus, a planning AI that can handle tactical targeting task but can’t handle politics probably “can’t war” going by definitions.