I'm a very happy vim user too. Sadly, short of commercial IDEs, developing lisp without slime is like developing Java without eclipse or intellij; it's just an inferior way to interface with the language.

For vim users, the two choices are vim+slimv and emacs+evil. I find evil to be closer to vim than slimv is to a good slime experience, but that can change. Also, if you have any vim plugins that you really depend on then slimv probably edges out evil.

On the other hand I started out several years ago with viper plus a bunch of hand-coded elisp functions to use my most commonly-used vim commands, so we've come a long way.