I’ve done some unscientific blind tests with nice modern class Ds on people who avowedly think all digital amplification is trash. 9 times out of 10 the reaction has been something like “hmm, how lovely, now when are you going to show me the digital amp?” After I tell them what it is, it becomes “oh yeah, I can hear it now. It’s fine I guess for casual listeners but clearly lacks X.”
People consistently report higher levels of satisfaction with more expensive wine versus cheaper one, even if it’s just the same wine rebottled. They’re not lying, our perceptions and preconceived notions genuinely affect the we experience reality.
I have noticed different headphones and using a class AB headphone amp vs builtin switching amps can significantly impact how long I can listen for... But that anecdotal correlation is not enough for me to assign blame to switching amps, it might just be crap switching amps... or switching amps with not enough headroom. The problem is class D is always used for cheap low power devices, so most bad experiences are inevitably going to strongly correlate with class-D - whether or not it's attributed to the fundamental property of switching vs just being a bit ... shit.