I have learnt Chinese and use it for work. Chinese characters have a personality of their own, you don't need to know the sound to remember them. You could even read perfectly Chinese and being unable to communicate orally. This very hard to understand or even believe when not experienced for real.
So yes, learning Chinese or Kanji has a steeper learning curve, but no, it is not that hard, and in fact from context and shape you can guess a possible pronunciation and meaning for unknown characters, sometime.
In languages where letters are mapped to sounds, even after phonetics changed a bunch, the written and spoken forms are huge cues to each other. (There are exceptions).
If a new word came to be in English and it was written, for example, "scaramara", two fluent speakers who only saw it in writing would probably guess similar pronunciations for it, to the point that one would understand the other's speech. I suppose something like that could not happen in Chinese. Is this assumption correct?