> there also many good counter-arguments.

Someone will develop it, and someone will use it for nefarious purposes - as one terrorist organization spokesperson once said, "We only need to get lucky once. You need to get lucky every time". If it is developed like this, in the open, at least we have a better chance to understand its capabilities and limitations, and to be better equipped to build countermeasures.

This is exactly the Prisoner's Dilemma, and also shows up in arguments why we need to build insane numbers of nuclear weapons. The logic rests on a faulty assumption that there is a binary choice - either you build it this way, or somebody else builds it this way. It doesn't consider any other possible paths that could lead to better outcomes.

There is an economic incentive to build it this way - for all the legal, and illegal applications it may be used. Movies can make stunts look more like the actors they portray more easily (this has existed for decades now), you can pay Tom Cruise to use his likeness on a publicity campaign without having to pay for his time. You can change your face, or your body, so you are not recognized.

It's a virtual certainty that a technology that can have a commercial application will be developed. I can't imagine a plausible path departing from this present where it isn't.

Yes but in the end it’s all about choice. It’s not some creation of chaotic coincidences and circumstances. It’s not some accidental discovery. It’s an active choice by whoever decides to make it for whoever purpose.

The argument isn’t matter of should it be made, but that the only reason it’s being made is because of people actual choices to do so. It’s not randomly spawning Into existence of its own free will.

An extreme version of this might be:

> I saw Bob about to stab Alice, so I beat Bob to it and shot her in the head. I did nothing wrong since the outcome would have been the same regardless of my actions.

On the bright side, you probably saved Bob's life.

Yeah, that’s correct. I wanted to match the original comment’s phrasing and thought it was clear from context, but I probably should have just used a different word to avoid ambiguity.