News for hackers. Stuff that matters.
Re:Apple’s big test of data integrity by jeffbee on Tuesday May 30, 2023 @05:27PM (#36131101)
How do they checksum 9GB of data and still boot promptly?
Re:Apple’s big test of data integrity by sillywalk on Tuesday May 30, 2023 @05:43PM (#36131326)
FTA: "This is the ingenious part: verification carries little overhead, as it runs as a rolling process once the top-level ‘seal’ has been verified. So verification is a continuous process as the SSV is mounted and accessed."
Re:Apple’s big test of data integrity by vlovich123 on Tuesday May 30, 2023 @07:03PM (#36132336)
Which presumably means Merkel Trees.
Re:Apple’s big test of data integrity by droffel on Tuesday May 30, 2023 @06:27PM (#36131872)
Merkle trees! You (simplified) hash the data at the leaves of the tree, merge adjacent leaves (concatenation for example), and hash the new leaves, until you're left with just the Merkle Root, a single hash representing the entirety of your data. Verifying the root is easier than loading the data itself into memory and hashing it there since you can verify it piecemeal and without loading it all into memory at once.
Re:Apple’s big test of data integrity by krackers on Tuesday May 30, 2023 @07:20PM (#36132499)
But regardless of the ordering which you read, don't you ultimately still need to read every bit once?
Re:Apple’s big test of data integrity by CJefferson on Tuesday May 30, 2023 @07:48PM (#36132751)
Each level of the tree contains a hash of the hashes from the level below, so to verify the top hash you need only hash all its children (which are hashes themselves). Then you can explore a child, continuing recursively. At the bottom you find a hash of an actual file (or part of a file), then hash the file to check it’s validity.

This allows you to only hash those parts of the tree you actually want to read.

Re:Apple’s big test of data integrity by viraptor on Tuesday May 30, 2023 @09:15PM (#36133339)
> Big Sur, Monterey or Ventura in default Full Security mode has verified every last bit of their 9 GB SSV.

In this case they're not selective, so the tree approach doesn't save the reads.

Re:Apple’s big test of data integrity by vlovich123 on Tuesday May 30, 2023 @11:23PM (#36134136)
If you want to verify the entire image, I don’t think you can get around reading the entire image. Because any part you didn’t read to verify the hash is a part that could contain corruption.
Re:Apple’s big test of data integrity by nomel on Tuesday May 30, 2023 @09:16PM (#36133347)
That's around 3s, for their slowest offering.
Re:Apple’s big test of data integrity by w-m on Tuesday May 30, 2023 @06:41PM (#36132087)
Don’t these machines have read speeds of like 5000 MB/s? So just read the 9 GB blockwise and apply a super simple hash. Should be done in two seconds.
Re:Apple’s big test of data integrity by viraptor on Tuesday May 30, 2023 @09:24PM (#36133408)
If I include the fs overhead, I can shasum a single 23GB file (likely not fragmented) in 46s.

That's uncached, so it's not that quick at boot. 9GB would take ~19s then. Block would go faster, but not an order of magnitude faster.

Re:Apple’s big test of data integrity by Filligree on Tuesday May 30, 2023 @11:43PM (#36134230)
Single threaded or in parallel?

It's a Merkle tree, apparently, so parallelizing the checksum should be trivial.

Re:Apple’s big test of data integrity by foobiekr on Wednesday May 31, 2023 @12:26AM (#36134454)
it's IO not CPU bound