When you use software volume control, you're not changing the volume of the speakers - you're changing the volume of the recording. If your speakers have any background hiss - and they do - then making the recording quieter makes that hiss easier to hear. What's more, even the recording itself actually inherently has a kind of "hiss" because, in any recording medium, there is a quietest possible sound.
You can hear it for yourself - turn the volume in VLC very low down, and your speakers all the way up to compensate, and listen to a record - it'll sound bad. The louder your speakers, the lower you'll be able to crank VLC, and the worse the whole thing will sound.
Making the recording louder is - theoretically - fine; it shouldn't make it sound any worse. But if the recording is well-mixed, there's no headroom to make it louder and it will distort - this is because the engineers make the recording as loud as possible to avoid exactly the above issue.
I assumed from the "not bit-perfect audio" that VLC somehow misses bits or something, as in, doesn't play all the music information, but from your comment it just sounds something to do with volume. I also don't understand a word of your second paragraph, but I won't worry about that—they sound like dark, arcane realms of knowledge safer to avoid.