Right, residents of adjacent (or even nearby) properties would have an issue with it. Same as if I bought an apartment building and started using it as a factory. It's exactly why we have zoning.

A whole rented building has the same problems (loud guests, people abusing shared spaces, etc) as a single apartment rented to short-term guests, but on a larger scale.