You're right, on the face of it it sounds absurd and by rights it should be. However, absurdity has never stopped mankind from doing horrible things. I for one don't see the gravest dangers in the tech itself, but in the stuff that people will (ab)use it for.

And, of course, should AGI (which won't be text continuation, by the way) ever actually happen, then all bets are off. I just don't believe we're in any immediate danger of that.