I’m not him but I read his comment as simply taking issue with the idea that it’s minorities in particular who don’t have time for that, rather than simply poor people. (The terms are not interchangeable, since there are many wealthy people who are members of a minority, and there are many poor white people)