Most people are irrelevant for commercial aviation. And there are entire sections of the market for which most rich Americans are irrelevant.
A supersonic jet with concorde-like economics in supersonic routes would be usable if it could fly subsonic at a competitive price on the way back, but that's not how physics works.
These overfly land which, with current aerodynamics, means a small plane to keep the boom down.
Demand on the Concorde's London-New York route got flattened by lay-flat seats. The Concorde was built to the standards of 1960s aviation comfort. Three hours in an uncomfortable seat vs red eyes both ways.
The regulatory environment is the economics. A majority of airline customers won't pay extra for supersonic flight, so whether it's worth having a separate model of supersonic plane that might represent 10-20% of plane sales for planes of that passenger capacity is down to how much it costs to certify a new model of plane.
https://newatlas.com/aircraft/nasa-50th-anniversary-us-comme...
There really is a "peak comfort" or "good enough" for most people, the metrics of cost and convenience can trump objective quality and performance.
Moving the amount of people we want to move, for the cost at which people are willing to endure, shook out the way it did and supersonic lost out.
For general aviation you just have many more challenges to overcome than say driving. It takes really special circumstances before it makes sense.
We won't have flying cars for the same reason we don't all have planes. They aren't required, few need them and they would be more inconvenient to operate than a simple car.