Otherwise land values rise too high and fast to establish rights of way.
The British and US rail systems were developed as industrialisation progressed, notably IR2 (coal, steam, iron) and IR3 (steel, electricity, precision).
The US rail system was further developed in the wake of depopulation, genocide, and land appropriation from the indigenous tribal peoples. Governmental land grants to railroads were a major funding mechanism.
Development of HSR has tended to be a Post-WWII phenomenon, originating in Japan, then France, Germany, elsewhere in Europe (though generally not the UK), and recently, China.
The usual factor cited has been density, which matters. But both national economic policy and rights-of-way acquisition costs are massive factors, which the more free-market, strong-private propert rights regimes of the US and UK resist.
Wikipedia's list of HSR systems excludes Switzerland, though lines are presently under construction there (notably GBT): including Belgium, China, Denmark, Finland, France, Germany, Greece, Israel, Italy, Japan, Morocco, The Netherlands, Norway, Poland, Russia, Saudi Arabia, South Korea, Spain, Sweden, Taiwan, the United Kingdom, the United States, Uzbekistan, and Turkey.
One reason existing tracks and routes don't run that frequently is that there aren't the passengers to make them worthwhile. Another is that those tracks are also used for freight.