They carry disease, you may not even realize it. They don't have to touch you or bite you to spread pathogens either. They can get into food stores, warehouses etc. and cause issues. Their fleas in particular are good vectors for spreading disease

Also, they can be very aggressive at pets, in certain circumstances, while I admit, this can be (at least in part) the pet owners fault.