It bugged me at the time, but I’ve come to accept he was probably correct. It’s not a good idea to bring attention to matters that might work against you. Focus on selling yourself, and treat your business associates as customers (in the sense that you work hard to please them, in exchange for getting what you want). http://www.paulgraham.com/judgement.html
It’s important to know how people in better situations open doors. Because you can play that same game, with crafty use of time and words. And you might have more time than it seems, even in the position you mentioned (though I would not ever say it’s a guarantee): between Uber shifts, I’d park in a random store parking lots and build things, partly to keep sharp, but partly to get a job at a certain place, since the work was related. I didn’t get that job, but I gained so much knowledge that turned out to be much less obscure than it seemed; I still use it to this day.
You’re right in general though. People on a strong upwards trajectory have different concerns.
Sanitary worker turned software engineer is a very powerful story that can help you get a job, since if you’ve shoveled shit, fixing a nasty bug or doing a menial task wouldn’t be beneath you, and you’d make sure you do a good job. Managers seem to like that.
When I was at my lowest point I was delivering sushi rather than waiting for a job that would look good on a resume. Most of the people in desperate situations like that aren't (don't have time to be) padding their resume with side-projects, and therefore aren't the topic of the original argument.