I don't think we disagree? My claim was that many bright low-income kids don't know about or understand the opportunities available to them. Obviously it's not because they're stupid -- we're talking about kids smart enough to go to Harvard -- so yes, I think the majority of the reasons are cultural and/or stem from lack of information.
You're going to assume that (1) they are just saying that because Marketers Are Liars, (2) you're not smart enough - those schools are only for smart people, and (3) you can't afford it anyway.
I know someone who didn't know the difference between an Associate's and a PhD when they were in high school, and didn't even really conceptualize the idea that college was a real possibility when they were a teen.
My point is, you have to shift your entire cultural context when talking about the poor and the uneducated: you - anyone who's not deal with them - has to get the narrative context they live in before you can dismiss their non-attendance at the top schools.