E.g., up until a while back, 32-bit ARGB as exchange format would've been sufficient, but is now insufficient to handle 10 bit channels and/or stuff like HDR. What if we get a standard for 3D images? It gets worse if you want to handle animated images as well, whatever exchange format you specify will break once a decade at least. And god forbid someone tries DRM for images again…
Winamp has a similar property. If in 2030 FLAC is replaced by some fancy neural-network based lossless encoder, it can still read the files and play them without touching the core of the application, because all that's needed is a single plugin.
Your description might help with one specific area (compression algorithms) but there are numerous other areas where you can't just use a plugin.
Modularity can be useful, but there's always a core, and sometimes the core needs a change.
And that's before we even get into security issues. Sometimes entirely new classes of security problems are discovered, and if they weren't known at the time of writing, how were the authors supposed to guard against them? How much can we fault browsers for not guarding against spectre/meltdown when they were first written and released?
Anyway, lots of stuff follows this 8-bit-per-channel api. Until you use RAW images or any number of higher color depth . Then you need 12 or 16 or 24 or 32 (or more) bits per channel.
Alternatively, they could be designed to support plugins.