Been there! :). Clojure was my first lisp and I was learning emacs along with it. Looking back, emacs was a really good investment. Also, the ability to develop and debug through remote swank connections is so useful. Slime(with paredit) and swank bring out the best of lisp development.
I spent substantial amount of time setting up every environment mentioned and ended up with Clojure-Box. I have not tried emacs before; ideally, i should be trying a new language in a known setup, but I prefer the stability of clojure box, so i am taking my baby steps in emacs too.