For an image gallery, the balance could be the opposite: a small (56k) download of a decoder allowing to show many large (a few megs each) pictures.
I wonder how good the performance of that decoder is, though; if the decoding delay is much longer that the network transfer delay, the approach becomes much less appealing.
It's also not common to need a ton of huge images all at once — in almost all applications you're either displaying thumbnails and JPEG is fine and will load faster thanks to the browser's preloader getting those requests in faster than the JavaScript can run or you're looking at something like pages where the latency is easily hidden by preloading.
That means that in most cases the main driver is lower network transfer and you need to be serving a LOT of images to outweigh the cost of having to support something new and complicated versus something as widely tested and supported as JPEG.
can is not the same as must ... there is absolutely the possibility of adding support as a binary implementation, especially with the relatively short turn around of green browsers. IE11 and Safari are the last of the old guard on this and mobile devices have a 2-3 year burnout in terms of support.
Not to mention it could be feature detected via browser, js and other means as an interim solution. Also, I'm pretty sure the implementation is wasm with a very thin JS shim, at least that would be my presumption as I'm not familiar with this format/project.
It's extremely ironic then that IE11 and Safari are the only browsers to support HEVC, the rest didn't bother because of the licensing issues (which BPG inherits) severely limiting any chance it has of ever being natively supported.
AVIF however may have a chance.
I think you answered your own question right there.
I'm not going to use a dumptruck to bring a single sack of sand to my garden.