People underestimate how new much of the Western US is. For example, Dallas only began expanding in 1891 after the railways were built, LA was a small town until the 1910s-30s era expansion, modern San Jose only formed in the 1960s-70s after absorbing dozens of farming towns like Alviso and Berryessa, Seattle was mostly sand dunes until they were leveled in the 1900s-30s).
Because of how new it was, most of the cities are planned primarily with cars in mind - especially after the 1930s era Dust Bowl Migration and the 1940s-60s era economic migration. Same thing in much of Canada and Australia as well, which saw a similar postwar expansion.
> before the NIMBYists stepped in
NIMBYism in SF only really began in the 1970s onwards.
While it is now elitist, it initially started out as part of the civil rights movement ("urban redevelopment" was often a guise for razing historically Black, Hispanic, and Asian neighborhoods in that era - for example much of Japantown/Fillmore) as well as the early environmental movement (eg. Sierra Movement, Greenpeace), which was opposed to profit motive compared to modern YIMBY+Greentech model.
But my comment was a bit tongue in cheek - it is mostly political dysfunction. Of course the US could find people willing to work for less than $400/hr or whatever, but there is an incentive disalignment.