Or maybe it haven't happen at all? "US air force denies running simulation in which AI drone ‘killed’ operator" https://www.theguardian.com/us-news/2023/jun/01/us-military-...

On the other hand, I'm not sure whether I would confirm active research work on apparently rather capable combat AIs right now, if I were some USAF official.

The original blog post [1] is a summary of the talks at the "REAeS Future Combat Air & Space Capabilities Summit"; the summary of the AI talk is 2/3 of the way through the blog post. So either:

* Col Hamilton leaked information he wasn't supposed to, and now the AF is trying to deny it

* The person who summarized Col Hamilton's talk was really confused about what was said, and the AF is generally trying to shut down wrong impressions

* Col Hamilton and/or the summarizer made things up. (Was the blog post written by GPT-4?)

[1] https://www.aerosociety.com/news/highlights-from-the-raes-fu...

Yes, if this happened at all, either the military AI used here is far, far more advanced and was able to synthesise the action from whole cloth (in which case they're not going to blab about it on the internet), or the option to kill the operator was explicitly put in as an first-class "lever" the AI was given, or the whole thing is PR fluff to "start a conversation".

It might not be real, but emergent behaviour in self-play is a thing [1], and it’s not that hard to imagine a simulation where some random action (destroy comms tower, etc) resulted in unbounded point scoring and became a learnt behaviour.

You’d have to have an enormously complex and detailed simulation, simulated repeatedly, for this to happen, but it’s not totally beyond the realms of possibility.

[1] - https://openai.com/research/emergent-tool-use, scroll to the end re “surprising behaviour”.