The Swiss did vote for and consequently built things like the Gotthard Base Tunnel.
The Chinese are also building a lot of rail infrastructure and are of a similar size to the US. But then they have higher population density. But it's probably more a question of political priorities than physical realities.
The US geography makes high speed rail at the Federal level economically infeasible. Air transport reaches everyone including Alaska and Hawaii and Puerto Rico.
Now air transport can reach everyone, but is also especially climate unfriendly. Maybe in the future there will be electric or hydrogen airplanes, but if there is an option to use rail or plane mass transit, rail will always have lower energy demands. But historically the US has low energy (and fuel) taxes, and that will favor the planes. EU on the other hand is now talking about taxing aircraft fuels and if that happens and the taxes are sufficiently high, trains will be more attractive, even if currently travelling long distances by rail in the EU is a huge PITA.
To put it another way, Europeans haven't built high speed rail from London to Baku for similar reasons. The tree lined cloisters where political theory makes all nation states equal is not a good model of physical reality.Swiss rail hasn't connected Bern to Baghdad.