There are all kinds of cultures ("communities of practice", is the term) that we find ourselves in. Academics is one of them, in the sense that being a successful member requires you to be proficient in things like making coherent arguments, arguing from evidence, and asking clear questions. If your parents were university faculty, for example, you likely have much more exposure to a world where this is the norm than if you're first-generation college bound, which will make it much easier for you to navigate that world. That doesn't mean there's something "wrong" with academic culture -- it evolved because of what its practitioners do.
To the extent that "university student culture" crosses over into "the culture of the well-off", though, I'm inclined to agree with you.
Maybe the academic culture is wrong? What if that culture is applying arbitrary standards that disproportionately filter out certain groups, even when those standards don't translate to the skills needed in the real world?