What you need to know of Java in order to use the libraries with Clojure and Scala can be picked up as necessary while you are learning those languages. Depending on your field and interest, though, it can't hurt to know Java anyway.