That's exactly how property ownership works in America. Cf. adverse possession, in which a landowner who doesn't visit the property often enough to notice someone openly and notoriously using it without permission loses the right to do anything about it and it becomes the new person's property.

(However, in Michigan, the term required to acquire title by adverse possession is 15 years.)