So if the vast majority of the work being done is fundamentally different, does it make sense to set policy based on the outliers? The work of the patent office is almost entirely focused on "common ideas".
NB: I don't know the answer to this, just pointing out the difficulty.
But judges and PTO examiners have been getting it wrong so long they don't even know what's right anymore. But the letter of the law is quite clear and the SCOTUS is starting to come down harder on the side of the law, especially since rulings like Bilski.
Generally, I'm against software patents due to their easily abused nature (to the point where I think sacrificing all software patents would be a net benefit), but I find it very hard to argue that the person who came up with that codec shouldn't be able to patent their invention. Yes, it's math at the end of the day, but it's still novel math. Putting existing things together in actually novel ways should be patentable.
Novel math is still abstract, and as such, strictly ineligible. Software is the same, in that it is purely abstract. It will take time for the courts to catch up to reality, but they have trended in that direction.
Whether I agree or not that, as you say, "...novel ways should be patentable." they are, by the letter of the enacting law, ineligible.
That it involves not just some singular "let's do an X" moment, but tons of math, hard work, novel techniques for better compression, faster decoding, etc?
I think you'll find this takes a little more work than "Hey, wouldn't it be great to make audio files smaller?"
That's for common ideas. "Let's make a Facebook for dogs", "dollar shave club", etc. But those are not patentable in the first place (and if they are in some country, they shouldn't be).
Here we are talking for invention ideas. Which are not plenty, and which often require not just insight that billions couldn't manage to have (the invention of the carriage wheel -- which the pro-Columbian population of the Americas managed to miss, even though they had wheel shaped items for other uses), but also millions in R&D (e.g. to design. build, make efficient to mass produce and test a new kind of plane engine or a more efficient battery).
In a lot of those cases where the invention scratches an itch there's very little "correct productization" needed. If you make a better battery material for example, you just sell it like everybody else sells batteries (plus some marketing for its longer life). Some things literally sell themselves.