(There are probably still compression technologies waiting to be invented; but applying them within the constraints of an existing format is hard or impossible, and creating a new format means you need a /massive/ improvement to make switching attractive for the consumer)
The LAME encoder was the x264 of its day, and there's forums where people who care test competing audio codec implementations and newer encoder revisions to see how progress is going just like they do for video.
I'm not saying it really matters to end users in a big bang wow sense that you can hang a blog post news item on, but codecs gradually getting better benefits everyone and is continually happening in the background.
For most lossy encoding formats, there are many ways to compress a single file, and in general, we do not know how to pick the best size/encoding quality encoding of all possible encodings. You can get better compression/better quality at the same compression by searching harder or by searching smarter.
Typically, compression settings deal with the 'searching harder' approach.
Searching smarter is what the successive generations of encoders do, but sometimes, this also is controlled by the compression setting because an alternative approach may be better in compression quality, but so much slower that an quality-wise inferior but faster algorithm is kept as an option.
Level 5 is better imho, as I guess most of us want smaller files not faster encoding. Computers today are fast enough to do this without a significant wait.
Also this: Created: 2007-10-19 Closed-fixed: 2009-01-02 Released: 2013-05-26