Are you just trying to take advantage of multiple cores on one physical box, or are you looking to spread work out over a cluster?

If it's the former, JVM threads should be able to take advantage of multiple cores, IIRC. Just read Doug Lea's book on Java concurrency:

http://www.awprofessional.com/bookstore/product.asp?isbn=020...

If you want to go over a cluster, there are a few options. If your problem can be done in map/reduce fashion, there is Hadoop: http://hadoop.apache.org or if you want a traditional RCP styled approach (eg, MPI), there are Java bindings for MPI: http://aspen.ucs.indiana.edu/pss/HPJava/mpiJava.html

In either case, Doug's homepage is a great resource: http://gee.oswego.edu/~dl/index.html

and the HPJava page might be of interest as well:

http://www.hpjava.org/