Scott Hanselman has said a few times that two layers of abstractions is generally sufficient. I'm not sure if he stole it from someone but using your example, that means to be a great Java developer, you should have a good understanding of the JVM, and an adequate understanding of C++. Do you need C++ to be a good Java developer? No. If you've got a decade of experience in both, are you a better Java developer than the guy who learned it in school and that's all he knows? Probably, but not necessarily.

In addition I think the argument falls apart the closer down you get. Granted I am no C expert but I don't think you need to know any assembly to be a very good C developer.