I lived out in the country for a while and we had a few trailers near us. That's not a problem, and in appalachia, it's sometimes all the better people can do. The problem is trailer parks, which are sometimes okay, but sometimes are just rural ghettos, with crime and crack and meth and general bad behavior pouring out.
You have a few strategies to "keep up the neighborhood". The upper class strategy is to price out the poor people. The middle class strategy is zoning, which amounts to discrimination of various kinds. The results are pretty much the same. The fact the rich people get accidentally what middle class people have to get by fiat causes political confusion.
That said, you can always build a barn and then put the trailer in the barn.
Yes they call it "taxes and zoning rules" but in practice it simply operates as a landlord charging rent and telling the tenant what they are and are not allowed to do with the landlord's property. If it walks like a duck...
Sorry about the noise, BTW, but the lot on the other side of yours is the perfect location for our CEO's private helipad.
(Incidentally, if you were really asking me about a shed, and not merely trying to bait us onto some slippery slope to anarchy... I rather like small cottages and would like to put one up myself. So I'd be pretty sympathetic at your zoning board meeting.
Ultimately, I'm guessing that zoning boards exist only because people were historically unwilling to just ask their neighbors first and abide by their preferences. And zoning boards can still be overridden if enough of your neighbors agree. It's called a "variance".)