Speech recognition is leaps and bounds ahead of where it was in 2001/2010, driven by a combination of deep learning and massive datasets. I'm actually working on a similar concept (speech recognition for Mandarin language learning) and even accented speech is becoming less of a problem.

I haven't tried differentiating tones for discrete words with the recent batch of generic models but I think it might still struggle. I suspect a fine-tuned task-specific classifier might be needed for decent results.