Sometimes the only requirement is to meet code, but even that creates issues when you want to build something for yourself.

We have some really far out places that are still "unorganized" in Ontario Canada and therefore no permitting req'd, but you're still supposed to follow the building code for anything over 108sqft.