Private toll roads don't get to discriminate against traffic. Internet providers don't get to discriminate against traffic. Why should payment network providers? Once you're big enough and important enough, you're quasi-state.
Toll roads change price based on type of vehicle and don't allow all types of vehicles. Discrimination.
In most jurisdictions internet providers absolutely discriminate on traffic to help resolve congestion, they are allowed to discriminate for VOIP under various circumstances. Discrimination.
Visa and Mastercard don't want a bunch of charge backs, some services are more prone to fraud than others, and they flat out don't want their brand associated with various things. Discrimination.
With few exceptions (race, gender etc etc), discrimination is a net good for the world. People should be allowed to decide who they want to do business with, socialize with etc.
I suspect they can't capriciously tell you that you're not allowed on the road because they don't want to associate with you, as you are a "scumbag."
Plus, banks aren't allowed to just close their eyes and allow all money movement via ACH.