Most people would agree with you. That's why pretty much every government has the ability to engage in eminent domain or compulsory purchase to take private property for the "greater good."

So, whatever is responsible for the lack of proper infrastructure, your explanation isn't correct.