> I'd assume it would qualify under whatever property right gives a business the right to refuse customers

So the Civil Rights Act of 1964, which restricted that right, would be a precedent from different protected classes.