That's literally what the Luddites were doing though. It's a reasonable comparison.

Luddite is usually used as an insult based on a misunderstanding of the Luddites. That’s the definition I’m responding to here.

I would disagree. Luddite, to me, is a negative and pejorative label because history has shown Ned Ludd and his followers to have been a short-sighted, self-sabotaging reactionary movement.

I think the same thing of the precautionary movements today, including the AI skeptic position you are advocating for here. The comparison is valid, and it is negative and pejorative because history is on the side of advancing technology.