It's because of the historical, and thus causal nature of the distribution of opportunity that we might have concern over the distribution.
But if we re-rolled the dice, it's not like the asian couple from around the world is suddenly going to have a caucasian baby. You being born where you were was a confluence of factors operating in a causal framework with some noise. A re-roll isn't going to make you a Hilton family baby.
Also, what a person is entitled to is a matter for society to determine. Society creates a construct that it feels is a wholesome lens or boundary, then it creates the structures to enforce it. There is no natural right to anything, including property, life, or even death (some states forbid you to choose your death). Society determines that property is useful, creates a structure to enforce it, and voila, you are now entitled to property.