That's one of the arguments behind Georgism. If someone is not able to own enough land in order to provide for themselves then working for someone else isn't actually a free exchange.