News for hackers. Stuff that matters.
Re:Universal algorithm set to boost microscopes by mehrdadn on Thursday August 29, 2019 @03:55PM (#20832989)
Be glad you're not taking RAW photos, or this isn't even an option. :(

Anyone have any suggestions for doing the same thing for video?

Re:Universal algorithm set to boost microscopes by ChrisLomont on Thursday August 29, 2019 @04:47PM (#20833498)
RAW usually compresses, just not as well as lossy. For example, the lossless FLIF image format usually compresses images quite well (and it's lossless - you can binary compare the before and roundtrip images to check).

For video, I suspect there's things like FLIF, but with interframe knowledge, allowing even more compression.

None of these should beat good lossy codecs, but they vastly outperform completely uncompressed data for the stuff people normally capture.

If you somehow capture true random noise, then these wont' work. But most recordings of interest are much more structured, hence compressible.

https://flif.info/

Re:Universal algorithm set to boost microscopes by mehrdadn on Thursday August 29, 2019 @05:59PM (#20834129)
Not really. Some RAWs from some cameras do compress, but not others. (Not sure why; maybe it's already pre-compressed.) As an example, I tried just compressing a random 30MB RAW from my camera (just a photo of jeans, nothing particularly unusual) with 7z on Ultra compression, and it saved—guess what—a whopping 2.2% of its size.
Re:Universal algorithm set to boost microscopes by remcob on Thursday August 29, 2019 @06:24PM (#20834306)
7z is a generic lossless compressor. These algorithms tend to specialize in exact repeating patterns in sequences. Good for english prose, but images tend to have non-exact patterns in two dimension. A dedicated algorithm like FLIF will be much better at picking those up. I would expect an order of magnitude difference in performance between 7z and FLIF.

For the curious: the way these non-exact pattern compressors work is by subtracting out the predicted pattern and then storing the remaining signal, which will be smaller numbers and therefore take less bits to store.

Re:Universal algorithm set to boost microscopes by ChrisLomont on Thursday August 29, 2019 @09:12PM (#20835170)
What is your camera? Multiply your sensor size by the bit depth, and I suspect you’ll have more than 30mb.

Also image compressors like FLIF vastly outperform general compressors on images.

Re:Universal algorithm set to boost microscopes by mpais on Friday August 30, 2019 @12:57PM (#20840334)
Some RAW formats are pre-compressed, usually with very simple (hence fast) algorithms, and some aren't even lossless (Sony ARW). It's often possible to further compress them with specialized tools.

Disclosure: I'm the author of PackRAW, a tool that does exactly that (https://encode.su/threads/2762-PackRAW)

Re:Universal algorithm set to boost microscopes by pletnes on Thursday August 29, 2019 @06:16PM (#20834257)
The RAW format is probably using e.g zlib for compression internally. You can check by calculating how many bytes in total a truly raw image is: 12-16 bits (1.5-2 byte) per pixel.
Re:Universal algorithm set to boost microscopes by the8472 on Friday August 30, 2019 @04:41PM (#20842498)
Many lossy video formats (e.g. h.264, h.265, av1) have a special lossless profile. But there are also dedicated lossless codecs such as ffv1.