The fact that state-of-the-art AI already fails at much simpler decisions.
In the case of the Black Sea incident, the potential consequences include global thermonuclear war.
It's always hilarious to see the confidently incorrect comments by a bunch of ignorant software developers. The Dunning–Kruger effect is on full display here.
Nothing is harder than self-driving cars. Nothing. We'll colonize Mars before we have a solid solution to that problem. Why? Self-driving cars have to coexist with human drivers and human infrastructure. Nobody in aerospace has that problem. If they did, they'd run screaming for the hills.
That simply doesn't happen when I fly my quads. "How would I write code to dodge an attacking drone? How would I modify my drone to drop a grenade or a Molotov cocktail, or otherwise cause a large amount of grief to people below? How would I build a SLAM model that allows the drone to do this without intervention from the ground?" None of these engineering problems bug me the way driving a car would. They are all addressable with multiple degrees of freedom, both literally and figuratively.
Meanwhile, on the road:
"Hmm, the light at this intersection is out. There's a cop with an angry look on his face, flapping his arms at me like a dying chicken. What does he want me to do, exactly?"
"Huh, here I am in Seattle, and it looks like they have chosen to mark the stripes on the road with some sort of paint whose complex impedance at optical frequencies is identical to that of rainwater. I'm sure glad I'm driving, and not my lane-keep assistant, which I had to turn off because it tried to steer me into the median the last time it snowed."
"Whoa, where'd that ambulance come from. The law says I have to move right, but the only way I can get out of his way is to move left, and in any case, that's what the car ahead of me is doing. What to do, what to do."
In most of the airborne scenarios you mention, doing nothing is a perfectly viable, fail-safe solution when confronted with a situation the hardware or software can't handle. If we approach driving that way, a few miscreants can brick an entire city, intentionally or otherwise.
I'm not surprised Karpathy tapped out, let's put it that way.
What makes you think that AI can't incorporate those into decision making? Pilots do these through instruments anyway.