News for hackers. Stuff that matters.
Re:Ask HN: What are the technical justifications for keeping... by Someone on Friday June 26, 2020 @09:36AM (#23651786)
So, to show thumbnails for a directory with 100 photos, you’re going to open and close 100 files, and read 100 1MB chunks of data?

That probably is doable on modern local hardware, but on network disks, it might still be slow.

Make that an IMHO not unreasonably large 500 photos and your local disk might be slow, too.

Add a virus scanner, and 100 photos might already be annoyingly slow.

⇒ I would try to find a way to store away the checksum somewhere in the file (say in an extended attribute, as a (modification timestamp, checksum) pair)

Re:Ask HN: What are the technical justifications for keeping... by loa_in_ on Friday June 26, 2020 @11:55AM (#23653404)
Normally to show thumbnails you need to parse the whole image anyway. Unless that is the format doesn't have downscaled data already (progressive jpeg/PNG)
Re:Ask HN: What are the technical justifications for keeping... by Someone on Friday June 26, 2020 @01:00PM (#23654203)
If you don’t cache thumbnails, that’s true, but that’s too slow, so you cache thumbnails.

If you do that, you only have to read the whole image the first time. Once you created the thumbnails, you only need to read them, and they’re a lot smaller, and can be put in a single file.

The proposed approach reads a significant part of each file to determine whether the cached thumbnail is still current. That doesn’t give a huge improvement over not caching thumbnails at all (factor of 5 in bytes read/network traffic, as a rough guess, likely less in disk seeks)

Re:Ask HN: What are the technical justifications for keeping... by lovelyviking on Saturday June 27, 2020 @05:34AM (#23660604)
It's a dilemma of memory vs computation. It depends on what we expect/prefer:

FM that takes a lot of memory or FM that use computation resources when required only. I tend to think that in general expectation from FM is to be very small in memory demands and take valuable computational resources only if it is absolutely necessary.

Re:Ask HN: What are the technical justifications for keeping... by loa_in_ on Sunday June 28, 2020 @09:01AM (#23668434)
My approach would work amazingly in ipfs where hash of contents is an intrinsic property of the file system
Re:Ask HN: What are the technical justifications for keeping... by loa_in_ on Saturday June 27, 2020 @05:24AM (#23660558)
You're right. I got a bit ahead of myself it seems.
Re:Ask HN: What are the technical justifications for keeping... by lovelyviking on Saturday June 27, 2020 @05:36AM (#23660616)
Thank you for thoughts though. Was interesting to read both of you.