I assume you know this, but Chinese is a tonal language. 'Mom scolds the horse' translates as ma ma ma ma. With nothing but tones distinguishing the various 'ma's. People who grew up with only tonal or non-tonal languages tend to do poorly with languages on the other side because they're extremely alien.

Like imagine not being able to use inflections to express emotion when speaking. That'd sound as alien to a non-tonal speaker as the concept of using inflections to express emotion to a tonal-speaker. Like think about how 'That's a great idea.' can be phrased in different tones to make it sound enthusiastic, insulting, a question, sarcastic, and so forth. You just can't do that in tonal languages.

There's also the side issue that most non-tonal languages tend to have a relatively close linguistic background. Russian has a massive amount of 'English' cognates (often from shared ancestors predating both), especially if you consider words that are basically cognates but have different stresses or maybe one slightly different vowel sound or whatever. But the number of cognates between tonal and non-tonal languages is far fewer. It makes them much more difficult to learn, at least for me.