Not all land let's you build there, and water is often a larger issue in the south west. But, the point is when land is cheap renting it is also cheap. Budget shelter at say 120$/month, food at 60$/month and living off of 600$ / month becomes far more reasonable. Transportation, and entertainment can be hard. At 55 walking a few miles to town and back with say 20lb of supplies might not seem so bad at 70 it's a much larger risk. Eventually most people flat out can't drive or walk so there are also end of life issues.

PS: Medicine is also a huge issue, but such is life.