The funny thing is that the person who worted top voted answer is not smart all. He might look smart for a newbie but the question is about tokenizaion and not about parsing.

So here is the reason: the top voted answers are wrong.