That's only true if the price of robots doesn't decline rapidly. Which (I believe) it is, and like virtually any other technology, will continue to do.
Even if not, we don't need that many people to open doors. So perhaps an office building will have 2 human janitors and 10 robot janitors; it doesn't really change the problem caused by souped up roombas putting the vast majority of janitors out of work.
I asked him why there isn't a robot to open and close the doors. He said that there is, but in this country its much cheaper to pay people to do it. (I think he said the cost of maintaining the door opening robots to be about $1m a year approximately.
So until we have a world where every country has the same salaries as USA/Western Europe/Japan, I think you will always be able to find work.
Even if that work should really be done by a robot