I agree that a lot of luxury is irresponsible, but at the same time I tend to view it differently depending on the situation:

An executive making a luxury purchase is different if their company is successful versus laying off lots of employees.

Buying a yacht can be part of an understandable (although expensive) hobby/lifestyle as opposed to something simply wasteful like buying solid gold toilet seats.

Building beautiful architecture, commissioning art, or adventuring/exploring (e.g. being the first person to climb X or to cross Y) can contribute to the vibrancy of society even if it isn't super practical.