Railroads (particularly high speed) need to be long and straight. What happens if people don't want a dozen trains roaring through their neighborhood every day? It's difficult to just go around it.

What happens if people don't want a railroad through their property at all? After all, once a railroad is sitting on your land, it's difficult to repurpose it for much else. The state can try eminent domain, but that takes time and lengthy court cases, and the costs add up quick.