Land specifically is kind of a special case, in that we can't just make more of it (to a first approximation). There is an ongoing opportunity cost to society to give you exclusive rights over a piece of land, and therefore it is right and proper that you make an ongoing contribution to society to compensate.