It's ultimately the passenger who would afford or not afford whatever rate. If drivers were truly contractors, Uber would just act as a a middleman and take a cut of whatever rate the driver set.

Edit - Or allow customers to set the rate, and again take a cut of whatever that was.