It has more vowels (and more frequent consonants), but tone doesn't change the word in English. It can change the meaning (denote sarcasm, sadness, or questioning) but using a higher or lower pitch doesn't change the way you'd write out the word. Remembering tones is like adding imaginary numbers to your math, it's something you never thought about that you need to now account for everywhere. Learning new individual sounds is hard (and I have to do it with Russian) but I don't think it's as hard as the tone business in Chinese.

As for filler words in English: I totally agree it's madness. The word "set" has like 20 definitions. But you can mess them up and people will still understand you. "I break into car" (the way a Russian learning English would say it) vs "I'll break into the car" are both understood the same; whereas in Chinese if I mess up a (to my ears minor) tone, the information is completely lost.