News for hackers. Stuff that matters.
Re:Bao: A verified streaming tool based on BLAKE3 by ebg13 on Wednesday January 15, 2020 @03:19PM (#22058042)
Is this functionally different than sending the hash of a block of bytes when sending a block of bytes?

> With a serial hash, the recipient would need to download the entire attachment to verify it

Rings to me as being trivially untrue, since, if the file is divided into chunks, you can just verify each chunk separately.

Alternately, can this be used to generate the complete file hash from chunks received out of order by a download using multiple parallel substreams without writing all of them to a block device in order first?

Re:Bao: A verified streaming tool based on BLAKE3 by oconnor663 on Thursday January 16, 2020 @01:10PM (#22067174)
The idea is that you have the hash of some file, and you trust that hash somehow. (For example, maybe you stored it locally in the past, or someone important has signed it, or it's included in the Bitcoin blockchain.) But now you want to fetch the contents of the file from some peer you don't trust.

If you just hash each chunk separately, you have a storage problem. Let's say you have a 4 GiB file, about the size of a DVD, and you use 1 KiB chunks like BLAKE3 does. The set of chunk hashes is 128 MiB. You can't store that much data in any blockchain, so maybe you hash it again and store the meta-hash of all the hashes instead (basically a custom 1-level tree mode). Then when it's time to verify, you fetch the set of hashes from the peer as a first step. This is all doable, but now you have to download 128 MiB of chunk hashes before you can verify a single byte. You can try to use a larger chunk size to reduce this overhead, but that increases overhead elsewhere.

Re:Bao: A verified streaming tool based on BLAKE3 by rfk on Wednesday January 15, 2020 @03:30PM (#22058135)
There are plenty of mischievous things an attacker could do if you just verify each chunk separately, including reordering or omitting chunks, and truncating the file.
Re:Bao: A verified streaming tool based on BLAKE3 by ebg13 on Wednesday January 15, 2020 @05:55PM (#22059738)
An attacker who can do those things could send you a different initial checksum as well. How does this solve that?
Re:Bao: A verified streaming tool based on BLAKE3 by rfk on Wednesday January 15, 2020 @07:55PM (#22060625)
Any sort of verification presumes that you have an initial trusted checksum against which to verify, so I don't believe this solves the separate problem of obtaining such a checksum.

IIUC your question here boils down to "what is the point of a hash tree?" as opposed to e.g. a list of individual chunk hashes. The answer is that a hash tree lets you verify an individual chunk by looking at the hashes of O(log(N)) chunks rather than having to look at the hash of every chunk.

Re:Bao: A verified streaming tool based on BLAKE3 by ebg13 on Wednesday January 15, 2020 @11:18PM (#22061654)
Thanks. That is meaningful, but I still don't understand how "the recipient can stream a video attachment, while still verifying each byte as it comes in" isn't basically also true for sequential hashing with periodic chunk hashes.