The True Cost in this specific case is (0..7 bits to fill a byte) + 2 bytes for the RST marker for each interval. 320x240 image and RST interval for each 240/8 block means 30 RST's so something close to 75 bytes "wasted", or, other way to look at it is ~3 bits per row.
The reason is that for the JPEG the Huffman tree is fixed and is transmitted with the image data so restarting is cheap; it does not start from scratch and build a new tree each time; each decoded symbol is also independent so you can stop and restart at any time you want. I don't have any ANS experience at all so won't say anything about that.. but it would be nice if some sort of way would be possible to isolate the ranges.
Good observations, btw. :)
FYI we actually do support hooks for malloc (JpegxlMemoryManagerStruct) and threading (JpegxlParallelRunner) precisely in order to be a good citizen :)
The topology is more about #cores/sockets, used by the benchmark_xl app to decide how many threads to spawn.
I agree adding Processor Groups makes sense, but Windows hasn't been our focus yet.
FYI it is indeed critical to parallelize ANS streams. Each 256x256 "group" can be decoded in parallel.
Just to clarify: I'm not criticising the JPEG XL code, it seems to be quite clean at first glance! I was merely making a general observation that would apply to practically all portable C-interface libraries, such as ZStandard, libpng, etc...
PS: I wonder if it would be possible to make a "portable library header" that all such similar libraries could share to avoid the rework on both ends (library authors and library consumers)...
> Windows hasn't been our focus yet.
That's a real worry. Not just because of the word "Windows", but what else it implies.
To provide some constructive feedback: JPEG XL will be yet another failure thrown on the trash heap of similarly well-intentioned formats. It'll die along with JPEG 2000, JPEG EXR, and all of the others. Don't be so arrogant to think that somehow your format will be special and adopted by the masses. The others were just as technically advanced for their time and they all failed.
Please, please, please learn this lesson, because this history of failure saddens me. The lack of a capable image formats compatible with all client platforms is what stops me as a photographer from sharing my images in full quality. In 2020, still, it is effectively impossible to share images in any other format than 8-bit sRGB JPEG across platforms. You can't email a HDR, 10-bit, or wide gamut picture to anyone and expect them to be able to open it and have it look even vaguely right. You can't post such an image on any website you don't control yourself. Even if you control the website, it will look wrong for 99.9% of users OR they'll just see the 8-bit sRGB version anyway, even if they have 10-bit HDR display.
It is literally only the walled garden of the Apple ecosystem that gets this right. Microsoft and Google most certainly doesn't. As an iPhone user, I can share a wide-gamut HEIF picture and 100% of the time it will look correct for other iPhone users. However, Apple "cheated" a little bit. When you send one of their HEIFs to any non-Apple-controlled app, it's silently converted to... 8-bit sRGB JPEG. It's just sad.
What everyone gets wrong with new image formats is this: They think that the problem that needs solving is the encoder/decoder. That's step one of many problems that users need solved. Image format developers assume their format will be "adopted", but there's no network effect. Why would anyone use a format they can't email, can't open on Windows 7, and can't edit in Photoshop?
If you stop with the JPEG XL library, with Windows compilation an afterthought, I absolutely guarantee that you will fail to gain traction and the entire effort will be essentially wasted.
To get actual adoption, you would need to:
* Most importantly, develop an image decoder plugin for Windows.
* Help Firefox, Safari, and other less common browsers merge support.
* Ensure that Chromium gets support, but I assume that's a given.
* Send pull requests to add JPEG XL support for all major open source libraries that perform image processing, such as WordPress plugins and server-side image processing utilities for websites.
* Develop a plugin for all major image editing tools such as Photoshop.
* Work with Adobe on Lightroom export support.
* Contact Sony, Nikon, and Canon and help them with in-body encoding support.
But even that's not enough. If you want 10-bit and the wide gamut (let alone HDR), you have to take steps to force support, or it won't magically materialise. You'll end up with two JPEG XL formats: The 8-bit SDR sRGB variety with 99.9999% adoption and the "fancy" JPEG XL that nobody uses and looks wrong if they try.
IMHO the only way to do this is to forcibly break applications that aren't processing the ICC profiles properly. Much like Google's Chrome experiment with randomising some TLS header fields to occasionally break middle boxes written by idiots that think version numbers are checked with equality, not inequality operators.
You'd have to do something drastic, like leaving the image planes unnamed and randomised so that literally the only way to decode the image correctly is via a compliant colour space transformation.
It sounds drastic, but other people have tried the easy way and failed. As I said: It's 2020!!! Email me a non-sRGB image that I can view in my mail client...
> That's a real worry. Not just because of the word "Windows", but what else it implies. We have put a great deal of thought into an adoption plan. "This time is different" because we actually provide value to existing clients/servers with JPEG, rather than giving them yet another bitstream to store, or lossy transcoding that just adds more artifacts. The lossless JPEG transcoding is a game-changer, as is the feasibility nowadays of decoding via WebAssembly - even with SIMD for speed.
> If you stop with the JPEG XL library, with Windows compilation an afterthought Good news, we are absolutely not stopping there. The project started in 2015. We're now moving out of the research phase and into productionizing, with integrations and plugins underway. Investing in such things too early would have meant less R&D.
> You'd have to do something drastic, like leaving the image planes unnamed and randomised Interesting idea, thanks for this suggestion.
Reminds me of a "lightweight" zip compression library that I recently used that was literally 98% overhead and 2% actual compression code. It had its own definitions of everything: integers, integer properties, memory allocation, streams (as in sequences of memory blocks, not even real I/O!), error handling, progress notifications, etc... Pages and pages of conditional macros and compiler-specific hacks. Just to call a single a function that took an array and returned an array. Ridiculous.
https://gitlab.com/wg1/jpeg-xl/-/blob/master/include/jpegxl/...
https://gitlab.com/wg1/jpeg-xl/-/blob/master/include/jpegxl/...
https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/compil...
https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_spe...
https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/arch_s...
That last one is trying to work out the NUMA topology in a portable way... good luck with that! That's not something I expected from a JPEG library. Like memory management, they'd actually be better off requesting thread-management function pointers from the hosting application. Otherwise, this would not be a "good citizen" in constrained environments such as embedded in a web server or database engine.
Random examples of silliness from just this one file:
They ignore the first 2 cores on 3+ core systems: https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_spe...
They'll be limited max 64 threads (Half the threads on the larger single-socket AMD EPYC 2 CPUs!): https://gitlab.com/wg1/jpeg-xl/-/blob/master/jxl/base/os_spe... because they're not using the Processor Group Aware API: https://docs.microsoft.com/en-us/windows/win32/procthread/pr...
Sure, 64-core monsters are still somewhat rare today, but 16-core/32-thread desktops CPUs are now affordable and the TSMC 5nm process will likely result in 96-128 core monstrosities this year. That's 384-512 threads for a dual-socket server or workstation! If you're making an image codec library "for the next decade", a little future-proofing might go a long way. Look at it this way: Image decoding throughput is one of the main performance issues I have right now with Lightroom and I work with "only" 36 Mpx images. There are 80 Mpx cameras just around the corner...
For example, there was a blog article a couple of years ago basically saying that the major Amdahl's law limitation to decoding formats like JPEG is the Huffman bitstream decoding, which is Step #1 in the process. The author pointed out that simply restarting the Huffman bitstream a few times for a typical PNG or JPEG file would increase the size less than 1%, but would allow truly parallel decoding for every step.
For all the apparent effort to make JPEG-XL parallelised by default, it seems odd that there doesn't appear to be any mention of parallelising the ANS bitstream decoding...