Thanks for the link. I think there's a lot of room for experimental, artistic, or even scientific specialty encoders. There's so much flexibility in picking out what's "important" in a signal.
The way you worded that immediately made me think of glitch art, particularly GIFs, and particularly situations where the glitch aligns perfectly with the content.
This is the post that got me thinking about it recently https://news.ycombinator.com/item?id=16034547
Source code for a lot of early encoders is available to study. Probably easier to understand than the modern implementations. LAME started out as a set of performance patches against the ISO sources until eventually being rewritten from scratch.
Toy MP3 encoders are not horrifically complicated, though ones that sound decent are. It's astonishing how much improvement there's been over the past few decades even with the same underlying format, modern 128kbps MP3s don't even make your ears bleed.