Tones are no big deal unless you think about them. Ever thought about your breathing? Tones are a bit like that. Just don't think about them, let the sentence flow.
Characters are the killer. Not hard, but slow and tedious. 2 per day... 5 per day... 10 per day... cram and 100 per day... but they're lost without constant reinforcement.
Then the thorn you were trying to ignore comes back, and idiomatic sentences (chengyu, based on 4 characters, 'long time no see' being of the form, or 'people mountain people sea' a super simple one) based on cultural or historical references come back, kick you to the curb again. Much as English idioms do.
And then speaking formally... different ball game. Pomp and circumstance rule.
But informal spoken Chinese is surprisingly easy to attain. Just get the accent down.
Going from/to the European languages to/from Japanese or Chinese (etc etc) take about 8000 hours to reach basic competency.