Definitely go for it, then, and try to do Common Lisp rather than anything else. You can always expand later to JVM-based Clojure, or indeed contract to Scheme, but it's good to start with the thing itself.
The world always needs more good Lisp programmers.
While Lisp is not core to what I do, neither is cooking, yet the company gives us classes in that, too. So I think I have a shot.