Capital-D Deaf culture does not as a rule view their lack of hearing as a horrible wrong that needs to be corrected for them to live a “normal” life. They believe that they are living a normal life, and screw you for suggesting otherwise, and many would tell you that they do not want to speak your inferior language anyways, because it is broadly technically inferior due to its inherent one-dimensionality: you describe things with a scalar pressure varying with time only, their language describes things with motions in three dimensions of space as well as in time.
This means that in the 95% case, the dominant attitude is not “I’m afraid,” it's “those folks are assholes half the time, why would I want to join them if I don't have to?”. It's a cultural objection rather than a physiological one.
I don't know the specifics of this claim, so I won't talk about the argument that I assume you're trying to make. However, the argument you're actually making here is completely untrue. Almost all deaf people understand and can communicate through language. In fact, if you put a group of deaf people together for long enough, they may well spontaneously generate a language. Sign languages are real languages with almost all of the features of spoken languages (and with additional features of their own). The idea that someone can only understand "language" if they are and to hear is completely wrong.
I assume what you're trying to say is that if you can't hear by this age, you will not be able to decipher sounds into language - I have no idea if this is the case. But this is an important semantic distinction (understanding language versus understanding language from sounds alone), and the fact that you seem to have inadvertently confused them seems relevant to the issue at hand: it is very easy for us to accept spoken language as a universal standard, when in practice, spoken language is just one (more common) variant of many different forms of language.
I'd be more interested in the source for their claim, than being pedantic about them missing "spoken" in where they clearly meant that.