Okay, but that assumes that control of AI is as trivially easy as controlling a knife. Given that this is one of the key unknowns, assuming that the AI never does anything that its controller doesn't intend is begging the question.

That aside, I can also think of many cases where I would feel less safe around a machine that had nobody in control of it, than one being adeptly controlled by an operator.

One important thing is that nobody has shown an actual existing AI to be not controllable, nor a way to create one. Until then it's an imaginary, non-existing risk. Of course as a fear and anxiety-based society we can also worry about those, but I think we should worry more about existing risks.