I imagine images appearing as fast as I can scroll, with enough visual clue to them to find images way way more quickly than with thumbnails today. As a start, thumbnails could use flif.
With FLIF you simply read the first N bytes of the full image and have a resonable preview. You choose how big or small N has to be, depending on the size of the thumbnail you want to show. Maybe first read N bytes for each image to get a quick but rough preview, then repeatedly read a few bytes more to enhance the thumbnail.
In practice, the simplistic resampling is not likely to be an issue -- of course you can create a malicious image which is white on the even rows and black on the odd rows, and then all previews would be black while they should be grey. But most of the actual images are not like that -- e.g. photographs. You can just decode at somewhat higher resolution and scale down from that. (You have to start from a power-of-two scaled image anyway.) Also note that Y is emitted at more detail earlier than chroma, so most of the error will be in those less important chroma channels. Other than that it's just Adam7 interlacing, but with no upper bound on the number of passes (so you could call it Adam-infinity interlacing).
Sure, for actual flif files you would not have a separate preview file.
The part which would really make sense for this would be something like an extension to <picture> or <img srcset> which would allow you to provide the byte ranges for each resolution so the client could issue a standard HTTP range request for the level of pixels it actually needs. You'd still have one file to manage and CDNs could cache it intelligently rather than lowering hit rates by caching different versions.
Judging from the video, something like that could work very well (and much better than with progressive jpegs).
The client could decide BY ITSELF to download only 2K for a thumbnail because it's currently on 3g.
It would take a few extra seconds when creating it, but could save both bandwidth and create a better user experience on mobile.
It'd be more accurate to compare it to a pre-generated resizing of the original image (at the same effective resolution as the "half downloaded" FLIF).
Though I'll gladly take your logic and call the first 1/64th of a progressive JPEG lossess (which it is, since at that point you only have the DC offsets of basic blocks, which undergo no lossy compression).
The PNG and GIF comparisons are unfair; they're the same low resolution as FLIF, but given nearest-neighbor upsampling instead of bilinear. Most web browsers use bilinear or bicubic upsampling for progressive pictures now, so FLIF isn't nearly as unique as it presents itself. Aside from that, progressive is only useful if you're on a slow connection or the file is broken; in other cases re-rendering multiple times becomes a bottleneck and battery drain. In general it's rare that progressive decoding is a win anymore (although progressive encoding of JPEG is always a win).
In another comment I pointed out that jpeg2000 can be encoded to any size, even 1k, even though the q-scale doesn't go that low, by using the target size parameter instead. It looks better than anything but BPG and is fully progressive. Of course, J2K is a dead format outside of DejaVu, PDF, and the medical field, but it serves as a useful baseline.
A comparison to JXR would have been nice; it's an open, patent-indemnified format that fully supports progressive decoding, and compresses about as well as WebP. That format's the biggest disappointment for me, I thought it had everything in place to displace JPEG.
BPG could be made somewhat progressive, but only by changing the underlying HEVC bitstream, while one of its goals is to stay as close as possible. The inclusion of a thumbnail is the only recourse.
But it's not just the interpolation that makes a difference: PNG interlaces full RGB pixels (all 3 channels at the same time), while FLIF gives priority to luma so you get a chroma subsampling effect on partial files. Also GIF only has vertical interlacing and PNG only does 7 passes of 2D interlacing, which means that for huge images, the first pass can still take a while.
Another point: I am proposing to use progressive decoding to avoid having to download the full lossless image (so a browser supporting FLIF would need some mechanism to stop downloading the image file when "enough" detail is available). Repeated re-rendering is not necessary. E.g. on a mobile phone you would rather just download some prefix of the file (the size of the prefix could depend on the available bandwidth) and render it once, at least until the user zooms in on the image, or saves or prints it or something -- then the download should be resumed in order to load more/all detail.
(Converting JPEG images to progressive format is one of the optimizations mod_pagespeed makes.)
http://storage.googleapis.com/marc-pres/boston-event-1012/im...
https://lh3.googleusercontent.com/-4tX6mfFoY4c/Vg7bg85RwcI/A...
It's been a long time but I think what I ended-up with was first scan MSB of Y DC, next six bits of Y AC. Then MSB of Cb and Cr DC, then a scan with a few bits of Cb and Cr DC and AC and finally the rest. The idea was that for a B&W, greyscale, or color thumbnail the same JPEG would be used but only the first N scans sent followed by 0xffd9 with a width and height in the img tag. Anyway, I can't be the only one that figured-out in the days of SLIP over modems that doing this trick was a good idea.
And my eyes - shimmering car grills and garden fences begone!
Now several problems:
1. If your texture is sampled roughly one pixel from it to one pixel on the screen, and if all pixels are read linearly ("horizontally"), then you are good with the cache, cause for the first pixel you've read the cache maybe cold at that address, but it'll load the next pixels just in case you need them, and here is the catch - you need to get always benefit from that - it's like always using your coupons, and deals, and your employer perks. So the CPU/GPU might read 4, 8, 16 who really knows (but you) bytes in advance, or around that pixel.
2. But then you turn the texture 90 degrees, and suddenly it's much slower. You draw a pixel from the texture, but then the next pixel is 256, 512, or more away ("vertically), and then next too, your cache line of 4, 8, 16, 32 or 64 read bytes that you did not used, and by the time you might need them again, the cache discarded them. Hence the slowness now - much much slower!
4. To fix it, you come up with "swizzling" - e.g. instead of the texture being purely scan-line by scan-line, you kind of split your image in blocks - maybe 8x8 tiles, or 32x32, and then make sure those tiles are linearly written one to each other. You can go even further, but the idea is that under any angle, if you decide to read, you most likely would hit pixels from the same cache line you've read before. It's not that simple, and my explantation is poor, but here is someone who can do that better than me: https://fgiesen.wordpress.com/2011/01/17/texture-tiling-and-...
8. But even with swizzling, tiling, whatever you call that magic to keep pixels together in any direction really together stops working as soon as you have to draw that big texture/image on a much smaller scale.
16. Say 256x256 would have to be drawn as 16x16 - And you say I don't need mipmapping, I don't need smaller versions of my image, I can just use my own image - well then nomatter how you swizzle/tile, you'll be skipping a lot of pixels - hop from here to there, and lost cache lines.
32. For that reasons mipmaps are here to help, but stay with me for one more minute - and see how they almost fix the shimmering problem of textures with "fences", "grills on a car", a pet's cage, or something like it.
64. And when you hear the artist ready to put real spider-man logo on the character made out of real polygons, and real grill in front of the car, made out of real polygons, and real barbed-wired made out of polygons very nice looking fence - then stay away, as these polys woulds shimmer, no good level of detail can be done for them (it'll pop) and such things are just much easily done with textures and mipmaps - it kind of solves a lot of problems.
128. Read about impostor textures.
FLIF claims it will beat the lossless compression ratio of JPEG 2000
[ed. image size → file size to emphasize that they had a target byte count for the purpose of comparison]
Edit: I was confused since with most coders you can specify precise mean squared error optimal truncation points (called quality layers) and it should have been very easy to obtain a stream of any particular file size.
Looks better than anything else but BPG at that size, to me, although FLIF obviously isn't optimized for lossy.
Basically, it allows you to use responsive images using 1 single master image. Makes for a really snappy user experience, and it's very easy to integrate into any project, new or old.
Basically, the last example shows that, if you want a scaled version of the image, you can simply stop decompressing. No need for multiple image files. Just create one with very high quality and decompress until you get the quality you want and scale it down with html.
edit: more info on the responsive side of FLIF: http://flif.info/responsive.php