It breaks down, IMO, for any non-circular definition of merit any time you have unequal distribution of wealth, even if the unequal distribution itself is due to prior merit. Inheritance is a dramatic example, but not the whole problem.

There's a reason I said “...at least, that's what the proponents of capitalism have always said.” I'm not actually endorsing the viewpoint as correct.