That's because we know what a "house" is - a bunch of private property with a wall around it (even if the wall is not locked). Usually comes in sets with other bunches of private property with walls around it. And we know the default - you aren't welcome in those walled forts unless you are welcomed in by the owner.

Not at all clear that files on a public webserver look very much like a private house.