Using Saliency in progressive JPEG XL images

Re:Using Saliency in progressive JPEG XL images by miav on Friday September 10, 2021 @05:05AM (#28479234)

It's very surprising to me that progressive loading techniques have been around for decades, but have barely been used on the web. Images often make for most of a websites content, size wise, and are frequently far larger than they need to be. Even those with generally good internet experience waiting for images to load for a noticeable amount of time. A 95% compressed version of an image is almost always good enough for a first glance and then the rest of the image can be seamlessly, progressively loaded. It's apparently been a ubiquitous problem for decades, with a known solution, yet it has not been applied. I'm curious as to why.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by JyrkiAlakuijala on Friday September 10, 2021 @11:08AM (#28482627)

20-25 % of JPEGs in the internet are progressive. MozJPEG outputs progressive JPEGs by default.

Safari chooses not to show intermediate results, but they are available with Chrome and Firefox.

The impact of progression on user experience is a complex question. It is for minimizing user experienced latency vs. (over)activating preattentive processing by images changing on the screen. Last year, Moritz Firsching (also the author of the blog post in question) made several improvements to the progression of traditional JPEG (libjpeg-turbo, Chrome, Firefox). Those changes -- in my opinion -- dramatically reduce the flicker and poor renderings that could occur from traditional JPEG progression, making progressive JPEGs more attractive overall.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by akie on Friday September 10, 2021 @05:36AM (#28479393)

Lazyness, indifference, time/budget constraints, and software that didn't bother to optimize for this. In other words: Hanlon's razor (https://en.wikipedia.org/wiki/Hanlon%27s_razor), more or less.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by specialist on Friday September 10, 2021 @09:27AM (#28481220)

Yup. Interlaced GIF (for progressive rendering) was the norm during dialup BBS stuff. As far as I know, web browsers always supported it. Dunno why the custom didn't make the jump from BBS to web.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by p0nce on Friday September 10, 2021 @05:35AM (#28479390)

Decoding a progressive JPEG when you have the whole stream is more CPU intensive and the decoder itself is more complex.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by joosters on Friday September 10, 2021 @05:46AM (#28479445)

Decoding JPEG images is not very CPU-intensive at all. Back in 1995, an ARM chip was fast enough to render JPEGs on the fly - i.e. apps didn't need to buffer the decoded versions, it was fast enough to draw them straight from the decoder when redrawing windows. (This was on RISC OS 3.6, IIRC)

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by teraflop on Friday September 10, 2021 @06:07AM (#28479570)

With battery-powered devices being so common, "fast enough" is no longer the only criterion for performance. Every cycle that the CPU doesn't spend idle subtracts from battery life.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by ben_w on Friday September 10, 2021 @07:28AM (#28480032)

Anything you could do live 26 years ago can be done today with an infinitesimal fraction of a modern mobile chip.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by thrwyoilarticle on Friday September 10, 2021 @09:04AM (#28480934)

>Every cycle that the CPU doesn't spend idle

Such as time spent networking?

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by gilrain on Friday September 10, 2021 @09:20AM (#28481129)

Network latency will dwarf CPU time in that case; the CPU can idle while waiting for responses.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by p0nce on Friday September 10, 2021 @05:48AM (#28479459)

It is more CPU intensive than a baseline JPEG.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by JyrkiAlakuijala on Friday September 10, 2021 @10:58AM (#28482492)

For the last 10 years progressive JPEG has been the fastest growing image format in the internet. Today, about 20-25 % of JPEGs in the internet are progressive. Mozjpeg outputs progressive JPEGs by default. JPEG decoding times are usually in the ballpark of 10 milliseconds, often considered an insignificant performance factor in the system analysis.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by sk65535 on Friday September 10, 2021 @06:06AM (#28479563)

rule of the thumb: progressive JPEG costs ~3x more resources (CPU, memory) than baseline JPEG.

With baseline jpeg, you can blit the decoded blocks to screen directly and forget about it.

For progressive, you have to buffer the whole image (in DCT format!) and do the idct / color-space conversion on all passes.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by JyrkiAlakuijala on Friday September 10, 2021 @11:02AM (#28482555)

To minimize additional computation JPEGs often use a limited number of scans, for example 5.

By default JPEG XL's progressive uses only two scans, 8x8 DC, and the transforms in the second pass using a 256x256 tiles in encode-time chosen priority order. This choise allows JPEG XL to do only one round of DCTs even in the progressive case. The 8x8 DC is interpolated using cheaper methods.

Because of the design choices, every JPEG XL image is guaranteed to be at least minimally progressive in the same manner, i.e., 8x8 DC first. Having a guarantee will make it more rewarding for system designers to focus on extracting some user-experience benefit from that feature.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by londons_explore on Friday September 10, 2021 @07:37AM (#28480103)

But, provided you don't need the intermediate results, you can rearrange the data back to progressive and then render it the simple way, for those times when CPU power is constrained more than the network. Total CPU use ends up being pretty much the same (the data rearrange step is rather cheap compared to the idct's)

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by sk65535 on Friday September 10, 2021 @09:20AM (#28481120)

if you re-arrange you have to wait for the bitstream to finish downloading, and thus lose the 'display something early on' feature.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by londons_explore on Friday September 10, 2021 @11:02AM (#28482547)

No... After all but the final pass of the image has been delivered you have enough data to start doing progressive rendering. The final pass contains most of the bytes, so you still get a pretty decent progressive render too.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by ajconway on Friday September 10, 2021 @07:47AM (#28480190)

On the other hand, it is ~infinitely less CPU intensive than decoding HEIF and AVIF with no hardware acceleration, and they were supposed to be the future of the web.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by sk65535 on Friday September 10, 2021 @09:22AM (#28481155)

AVIF is equivalent in decoding complexity, even in software. It's not order of magnitude different. Encoders OTOH can be as slow as you want them to be.

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by bellyfullofbac on Friday September 10, 2021 @08:13AM (#28480416)

Progressive JPEG would also be cool if bandwidth costs matter, nowadays a lot of images on the web have more pixels than the screen is able to display, so they are downloaded and downscaled. Or you have the HTML "imgsrc" attribute for "retina" images.

Imagine a file format that says "Byte-ranges 0-n will give you the image for a resolution of 640x480*. Bytes n+1 to m will give you the rest of the pixels for a resolution of 1440x1080, bytes m+1 to the end will give you the pixels you need to add to the 1080p version to get a 2160p version".

So the browser would decide what it needs and just requests the file up to that byte...

* The low-res 640x480 version might be all the browser needs if the person who designed the web page just wants the image e.g. as a small headline image in a news site column that is 640px wide

[ Parent | on HN ]

Re:Using Saliency in progressive JPEG XL images by jonsneyers on Friday September 10, 2021 @01:44PM (#28484881)

Basically that's what you can do with a progressive jxl: the frame header includes a table of contents that can be used to infer the byte ranges needed for a progressive preview corresponding to 1:8, 1:4, 1:2, and 1:1 resolutions.

[ Parent | on HN ]