My wife is a non-native speaker of English. I understand what they're talking about but they're mixing up two separate things--language usage and accent.

She definitely has an accent but it rarely is an issue. (There are a few words that mess her up badly, though.)

Much more of an issue is language usage. She has learned the words she needs to communicate (but will often be tripped up by uncommon words) but fares poorly with the little words that glue the language together. A recent example, "I eat myself" instead of "I'll eat it myself". The important concepts are in what she said, but with the glue missing the meaning changes greatly. Another weak speaker would probably understand it fine as the important concepts are there and they would arrange them to fit (but occasionally misarrange), but a native speaker not used to it very well might have trouble.

There's also the issue of sentence complexity. Think of books for little kids, how everything is broken up into simple thoughts. If their language uses the same sentence structure as we do it wouldn't be a problem, but if it doesn't they won't be able to properly build a mental map of what the sentence is saying without bits hanging off until the end.

I used to say that if you told my wife that if you can get to it in one minute there is a million dollars in a box on the back right of the second shelf of the refrigerator that she would have no hope of getting it, but if you told her there is a million dollars in the refrigerator, on the second shelf, in the back right in a box she would go right to it.

Note how it's easier for us to process the first way of saying it, the second feels wrong. For her, however, the second is normal, the first leaves too many hanging bits and she wouldn't be able to follow it.