I've conducted several A/B tests back when I had nothing better to do and it was a very education experience. I used a recording of a harpsichord, using lame V5 mp3s, and came away with a statistically significant result (I remember taking 14 trials, though I don't remember what my actual true positive rate was)

After all of that, though I had double blind evidence I could hear a difference, I knew I didn't really care about that difference. I couldn't even describe the difference - it was almost subliminal. And that was with a harpsichord, which is a known difficult sound to encode. I've been a lot more comfortable knowing lossy encoding isn't destroying my music enjoyment since those tests.

There are different things that could be found through tests, but I don't think I've seen tests being really extensive.

The interesting questions to me:

- Can a difference be heard?

- If yes, are people able to reliably identify A vs B (as in, can people identify whether it's A or B rather that whatever they listen now is not the same as the last they heard)

- If they can identify a difference, which sounds better to them?

- Do different headphones give different results?

- Same question with headphones vs speakers?

The answer to the last two questions is easy - yes to both.

There are absolutely huge differences between budget consumer and high-end professional equipment. There are also huge differences between high-end audiophile and professional equipment - the former being designed to flatter and enhance the sound, the latter being as neutral as possible.

A good few years ago a high-end audiophile CD/DAC maker was caught deliberately rolling off the top end on their equipment to make it sound smoother and "less digital."

It wasn't accurate, but a lot of people liked the sound enough to pay five figures for the boxes they made.

Professional audio has a different goal. The aim is to create a mix that "translates" - sounds good on as many different systems as possible, from budget earbuds to audiophile.

For a long time studios used a pair of shitty Yamaha bookshelf speakers called NS10s next to their ultra-expensive reference speakers. NS10s were almost the worst speaker ever. But if a mix sounded good on the NS10s, it sounded good on anything.

Blind testing is good for the first two questions, but it's next to impossible to blind test headphones.

The question of preference, though, is what matters, and that's where I am now firmly convinced lossy compression is fine for me: the sort of differences that I could hear made zero difference to my enjoyment of actual music.

Same. NPR Music published a quiz of this sort a few years ago (https://www.npr.org/sections/therecord/2015/06/02/411473508/...), and I failed pretty hard. I learned to play piano by ear too, so it's not like my ear is crap.

These aren't songs I normally listen to, but I did this quiz just now and even with laptop speakers I picked 3x uncompressed, 2x 320kbps MP3, and 1x 128kbps MP3.

128 is usually pretty easy to pick out, once you know what to listen for. There's no soundstage. They just sound flat.

The tricky part is distinguishing between 320 and uncompressed. 320 is pretty darn good. Even when I can hear the difference, if it's not a song I'm familiar with, I can't tell which one is better. It's just different.

Ha. That was interesting. I got 1/6, though out of the 5 I got wrong, I picked the 320Kbps MP3 for 4 of them. This backs up a few blind tests I did when I was younger, though then with music I was familiar with: 320Kbps MP3 and uncompressed WAV were typically indistinguishable to me, and, when they were audibly different, I still couldn't say which was better and which was worse.

The one I got completely wrong was the Jay Z track. Even re-listening knowing which is which, I still can't tell the difference :( - but this is a style of music I rarely listen to, and maybe that's the reason.

It's also a fair few years since I was 30, and that could be an alternative explanation.

(I'm actually quite surprised I did as well as I did, as the 128Kbps versions weren't obviously worse, and it did take a few listens on headphones. I should probably do another blind test with a 2003 build of LAME or whatever, not so much to compare MP3 vs WAV, but more to measure how much my hearing has deteriorated over the years...)

Ha, I just picked 5/6 the 128Kbps tracks. Maybe my ear has become accustomed to the compression artifacts and misses them when they're not there!

This is apparently a thing, that people will prefer what they’re used to and these days that’s low bitrate audio played through phone and laptop speakers...

I question how often people regularly listen to music through laptop speakers. Isn't it mostly through earphones or headphones?

Not necessarily, lots of students don’t speakers hooked up to their laptops, abd I see kids listening to music on their phone speakers all the time

This was fun, thanks for this.

Doesn't that test just intentionally choose bits of music that are hard to discern?

I did these kinds of tests in my early 20s (not sure how it would go now) and completely destroyed them. (I shut many many people up doing these tests, actually.)

> I barely ever can. > It's not something I'll notice when listening for enjoyment.

Pretty much tell the story.