Being born earlier is not nearly the only thing they did. They also (almost always) bought that land, (almost always) paid continual rent to the local government for the use of that land, improved and maintained that land, and finally, reached a decision that they needed to sell that land and improvements to someone else.

How do you feel about rent control? That gives someone who was there earlier a market distorting deal just because they were there first.