We also don't have an incentive to write programs that grant access to people who know a special incantation like '() { :; };' and yet we do. Point being we can fuck this up very easily, like any other bit of software.

I myself am on fence, on the one hand it is a bit like worrying about over population on mars, and there really is nothing we can productively do right now to research AI safety, the best thing we have are convolutional long/short term memory nets, and there's not much you can say about it.

But at the same time worrying is a great move if the alternative is to just wait until we can do something.