If you go through the wikipedia unicorn list [3], you'll find many founders that appear to be from middle or upper-middle class, but also many founders from lower-middle class as well.
[1] https://en.wikipedia.org/wiki/Brian_Chesky
[2] https://en.wikipedia.org/wiki/John_Collison
[3] https://en.wikipedia.org/wiki/List_of_unicorn_startup_compan...
Mark Zuckerberg: father was a dentist; solidly upper-middle class Bill Gates: mother on the board of IBM, upper-middle class
I won't try to talk about anyone except the Collison brothers, and given that I was born somewhat before them in the same country, I can assure you that they were solidly upper-middle class by Irish standards. Also, please note that Patrick Collison attended MIT as an international student, which is not something that poor people do.
It is a leap though. Bezos, Gates, Musk, Buffet, Page, Brin, Zuckerberg all started as regular people and became ultra-rich through their work and ideas.
But it's not a complete fallacy either. If you are a real estate owner in New York City, your assets continue appreciating at a good pace, well above inflation, and you have a vested interest in NIMBY, which does indeed restrict social mobility.
Social mobility, or social mobility opportunity are harder to measure though, so a lot of people are ok with using the inequality as a proxy. I do agree with you that it's an imperfect proxy, and focusing on it can have negative side-effects.