Maybe "hard" isn't exactly the right word, but it's certainly different. The sounds aren't hard to make, but using the wrong tone is like using the wrong vowel. You _have_ to speak each syllable with the correct tone or people won't understand you. This can be frustrating when you're learning the language because the difference is very subtle if you haven't grown up speaking a tonal language. If you say something that sounds correct to your ears, but native Chinese speakers give you blank stares, it's not that they're being obtuse or dense. They really have no idea what you're trying to say. Also there are relatively few sounds in Chinese, so if you make a mistake, you're likely to actually say something different from what you intended.