I get your point though, someone who grows up exposed to the inner-workings of business has a head start compared to someone without such exposure.
Kid has rich parents -> has privilege compared with those that have poor parents
Kid has non-abusive parents -> has privilege compared with kids with abusive parents
Has parent with some skill or trait to model -> privileged compared with kids that don't who in turn are privileged compared with kids who model bad behaviour from their parents.
Kid has parents -> privileged compared to those without parents.
There lots of things that make you who you are that you don't immediately consider to be contributing to your relative success in life, that's what the concept of privilege tried to remind people of.
We could come up with a million scenarios.
We should try to give everyone privilege.