That's a good point. You see phrases such as "consent shall not be unreasonably withheld" in contracts. It's common in leases, where you can sublet with the consent of the landlord. Court decisions in this area are not consistent.