I personally prefer working on a command line because it's _more_ discoverable to me than most gui's. This did take some getting used to though. The approach is different.
The worry over making dangerous mistakes is understandable. That said I have made very few big ones after a bit of learning, and it has also helped me _avoid_ making dangerous mistakes by facilitating the automation of otherwise delicate or involved tasks which would be unpleasant to click and type through.
I think the real usability issue of CLI is that their potential is handicapped if you don't know how to compose these programs, etc. In other words, if you don't know your way around a Unix command line, you're not going to be able to make use of the tool that assume that knowledge.
I think this is different than what you're saying, but maybe I'm just repackaging your points?
Pain points for me are discoverability and ease of making dangerous mistakes. I can't remember the command for every useful utility and it's very easy to accidentally break lots of things. Great power comes with great responsibility and all that.
What tools do you use for auto-complete-ish features on the command line? or anything to generally make using it easier.