But if you built a major infrastructure that could make SF or LA a morning commute, it'd sure do some interesting things to population distributions and land prices.
This is what we are competing with: http://3.bp.blogspot.com/-KM7B_auaBgE/UNr_iYv1v-I/AAAAAAAAAY... This is unlikely to get us there: https://upload.wikimedia.org/wikipedia/commons/c/ca/Fresno_C...
Shinkansen between Tokyo and Osaka is the busiest rail line in the world and there are a lot of stations between these two cities. The fastest train, Nozomi only has 6 stops and runs every 15 minutes.
The market size for SF to LA and back is not large enough to justify rail with no stops. Its not even close, even assuming monopolistic market shares. And airlines have shown extreme willingness to compete with rail on price grounds, and they could take half that market easily even with trains going top speed.
Sorry if that burns your dreams of fast rail to the ground, but it is objectively the most financially responsible way to build. It can actually pay for itself that way, whereas it would be America's biggest boondoggle if they took your advice.
https://traintimes.org.uk/KGX/EDB
These are high speed trains, but Britain hasn't upgraded the tracks, so they run at less than full speed. But the door design and number of passengers is what matters for the duration of a stop at a station, and as you can see, it's around 1-3 minutes.
It's certainly a factor, but it's not like the long distance Amtrack train I once used, which stopped for 5-15 minutes at most stops.
Traveling from city center to city center is also a far less stressful way to travel, and you can do things like work on a train that make TAXI -> AIRPORT LINES -> TERMINAL WAITING -> PLANE -> BAGGAGE CLAIM -> TAXI annoying.