You also gave up a lot of comfort to ride the Concorde. That would be even more true today with excellent business-class options on most airlines and many world-class first-class products like the Etihad A380 “Apartments”. Are high end business flyers or rich tourists really going to trade that to save 4 hours between New York and London in high enough numbers to sustain a regular route?
There is also the exclusivity factor, but that is already highly served by private jets. Then finally, just the experience. But is that enough to fill the seats every day?
Supersonic flight is a really cool idea but the reality involves a lot of tradeoffs. When it comes down to people actually buying tickets and looking at the customer base, I just don’t see it as some slam dunk product.