Emacs out of the box (install ruby-mode) is not even close to RubyMine. Now, if I spend many hours, and I value my time a great deal, I can get emacs kinda close to the features of intellij.
Frankly, I'd rather pay $60/year than spend many hours trying to get emacs close to as good as RubyMine.
When in Java, there is a mishmash of supporting file types, and it integrates them very well compared to its competitors - jump between .java, .jsp, various .xml for Spring, Hibernate, tag libs, etc etc. If you are "just" programming in a programming language (which is a lost utopia in for J*EE developers), much of this functionality is not of much benefit to you, but you might still find the IDE valuable.
Yes, it's quite good. Visual Studio is also very good in some aspects. And you can also get a lot of mileage out of Eclipse. Also try some Smalltalk environment, you'd be surprised.
The thing is, there are things missing from Vim/Emacs that are not easily added as ad-hoc plugins. Not merely this or that feature: what is mostly missing is a coherence in supporting a specific language.
And once you get ReSharper, also from JetBrains, Visual Studio becomes quite good, too :)
I'm working on a personal project in my spare time which I can use it on. (Except the project is using Common Lisp so it might not be a good match.)
I forgot to mention in my original post that I've also spent time with Unity3D recently although for some reason I didn't think of it when thinking of IDEs I had used in the past.