There is a big gap between being functional in a language and perfecting it.

Each sound cue not mastered hinders information redundancy and thus increases risk of not being understood. It probably seems like percolation: small crevices that suddenly reach a critical point where comprehension is lost at the receiver side.

Also way before that point, the receiver needs more subconscious effort and thus fatigues itself parsing the message.