The grammar is not that simple. I think that misconception is caused by people noticing that they don't have to memorize conjugation tables like when learning fusional languages, but then they fail to realize that the complexity has not disappeared, it's just encoded in word order and choice of grammatical particles instead.

There's actually an NLP shared task challenging researchers to automatically detect grammar errors learners make when writing Chinese: http://www.cged.tech/

I know that it's not so simple. But at least for a basic conversation, grammar it's not the biggest roadblock.

^ this. I find it a bit annoying when people tell me that Chinese "has no grammar" because it is not inflected.

I have been study a bit of Mandarin and the tones are hard, but when you get rid of them it's not so hard. The main difficult for me in spoken Chinese is to learn the huge number of homophones and some idiomatic phrases. On the counterpart, there is almost no grammar. Reading is hard (a normal educated person know around 10/12k ideograms, people with high level education are over 20k), writing is even harder because you need to know the exact order of the traits. I know that also overseas chineses frequently forgot how to write in ideograms.