Lots of types of access come with conditions. This is true of both publicly and privately owned land. Huge amount of public land are restricted (Salisbury Plain for example), and we have good access rights to some private land (most of Scotland). In a small country there needs to be some rules as everything is so close together. It is not as clear cut as you suggested.