Here in Taiwan, I meet tons of Mandarin speakers learning Japanese. In general, I'd say it's pretty easy for them. There are a limited number of sounds in Japanese only a few of which pose difficulty for Mandarin speakers and the kanji are a slightly simplified version of Chinese characters.

There are a lot of cognates between Chinese and Japanese, there's a good deal of overlap in terms of history, food and religion and best of all there's nearly as much Japanese media available for those who want it as there is English media.

However, the Japanese people I've met here studying Mandarin usually have a tough time of it. They can usually read pretty well, but their pronunciation and general communication abilities tend to be far worse than those of the students of every other nationality I've met at the language centers. The difference was absolutely shocking when I first started studying here.

One sorta starting point is the list of average TOEFL scores by native language of test-takers, which should tell you who finds English easy and who finds it hard. Page 9 in the link [1]. Of course, they say "use of average TOEFL scores to infer the overall English-language proficiency of a population or of a country is not appropriate due to a variety of variables that influence the test-taking population" =)

20 languages whose speakers get the worst scores: Lao (56), Bemba (58), Wolof (61), Mossi (62), Javanese (64), Khmer (65), Japanese (66), Arabic, Fula, Uighur, Uzbek (69), Burmese, Hausa, Kurdish, Vietnamese (70), Kazakh (71), Amharic, Oromo, Somali, Thai, Turkmen (72)

I wonder how much of this is due to how "complex" your own language's morphology and syntax are. For example, Mongolians tend to be better at English than either Japanese or Koreans who have studied for the same amount of time. And Mongolian's morphological and grammatical rules are much more complex than Japanese or Korean. (Anecdotally, Mongolians also find Japanese and Korean fairly easy to learn due to the structural similarities [2]).

For example, all those languages mark the role of nouns in a sentence by tacking on extra syllables ("particles"/"case markers"). For any given particle, Japanese invariably has one form, Korean one or two (some change if the word ends in a consonant), and Turkish usually two (vowel harmony). And the forms with pronouns follow the same rules. But Mongolian regularly has as many as eight forms depending on vowel harmony, what kind of consonant the word ends in, random rules for inserting "secret n"s and "hidden g"s, plus the forms for pronouns are irregular, plus there's rules for dropping vowels sometimes.

[1] http://www.ets.org/Media/Tests/TOEFL/pdf/test_score_data_sum...

[2] http://www.nytimes.com/2003/10/12/world/mongolians-and-korea... --- though contrary to what it says, linguists don't actually believe the two languages are related