Its within the training set I believe. The AI is being trained off compressed audio
I was listening to a podcast last year (forget which, or I'd attribute credit) and one of the tells AI researchers look for when trying to determine if music is AI generated or not is the compression artifacts. AI likely has gotten better at hiding it since then, but AI would include compression artifacts on the final product because of the prevalence of it in training data.
The problem is, I just... don't hear those things. I used to listen to 128kbps MP3s all the time and legitimately not understand wtf my audiophile friends were on about when they would say the quality was terrible. I come from a very musical background and am at least quite close to "perfect pitch", but apparently my ability to interpret sound as music has come at the expense of recognizing the sound for what it really is. :/
I think some people just can't unhear the compression artifacts. The equipment matters a lot. Bluetooth headphones from a phone, mp3 vs flac pretty much indistinguishable. Same as those little ipod headphones.
Once you get to DAC and IEM territory, the difference is quite a lot more noticeable. So for a lot of us it's not just file format but also the equipment to take advantage of it.
1. Usually weird little "flickers" of sound - like how a candle gives you mostly constant light but there are little flashes of bright and dark. AI generated music tends to have little distortions scattered throughout like that. As the models have improved they're less obvious but still absolutely there if you try to listen to any individual 'instrument' or 'track'
2. very low dynamic range - loud sounds and quiet sounds are much more similar volume than you'd expect from most real music, and things that are supposed to fade or taper-off in volume are unnatural
I feel like I don't quite know what to listen for here.