But some people don't have those so this is just unfairly disadvantaging the people who do.

This is the same reason why murder us illegal even though most people wouldn't do it anyways.