English prefers to borrow words from other languages. Curiously, it has borrowed hardly any words from Mandarin Chinese.

Besides that there had been much less contact between English speaking countries and China than to, say, German, French or Spanish speaking countries (at least until the Chinese economy began to boom) there is another reason that I can imagine. Since the Chinese dialects/languages are tonal languages their sound is much more foreign to the ear of people who didn't grew up with a tonal languages. Indeed people who learnt to speak one of the Chinese dialects told me one of the first exercises is learning to even consciously perceive the different tones.