The only people really limited in who they can see are those on public insurance like state coverage Medicaid for the poor or federal Medicare for the elderly. Those programs usually only pay a percentage of market value, and thus many doctors won't take them because they often lose money on the transaction.
There are many places in the US where the closest doctor is a few hundred miles away, and many more where the only doctors within a few hours are primary care physicians.
Although this statement may be true, it may also be irrelevant if there are no people in those places. That said, I'm not just being pedantic: my point is that lack of availability is far more likely to be a result of low population density than of being in the US.
Moreover, your comment fails to respond to the original point, which is that of affordability/availability to families with middle income. Poverty combined with low population density would likely result in even less availability, but, then, poverty is incompatible with "middle income".