News for hackers. Stuff that matters.
Re:BitTorrent v2 by johnisgood on Monday September 07, 2020 @05:20PM (#24403017)
Why not BLAKE3? I am really curious. I mean, since it exists, why not that over SHA256? Because it is relatively new?
Re:BitTorrent v2 by loeg on Monday September 07, 2020 @05:37PM (#24403138)
I would guess that the reasoning is similar to why Git is moving to SHA256 (from SHA1) rather than to BLAKE3 — SHA256 was around 5 years ago and the major design change has been in the works for a while (BEP 52 dates to 2017). BLAKE3 (2019) would be a fine choice today.
Re:BitTorrent v2 by johnisgood on Monday September 07, 2020 @07:43PM (#24403851)
I see. Thank you! By the way, I have not thought much about it, so in case you may know: would not it be possible to implement this in a way that allows swapping the hash function? So for example when we run into issues with SHA-256, change the hash function to something else.
Re:BitTorrent v2 by loeg on Tuesday September 08, 2020 @01:03AM (#24405355)
We already can: the "swap" will just be a v3 along the same lines.
Re:BitTorrent v2 by johnisgood on Tuesday September 08, 2020 @12:19PM (#24409884)
Yeah, but would not they have to create v4, v5 and so forth every N years, for different hash functions?
Re:BitTorrent v2 by loeg on Tuesday September 08, 2020 @01:23PM (#24410558)
Sure, but this is not any more expensive than any other versioning scheme you might be thinking of. Consider also that they got 19+ years out of v1, and that there is no reason to believe SHA2 will be broken faster than SHA1.
Re:BitTorrent v2 by johnisgood on Tuesday September 08, 2020 @01:58PM (#24410919)
Probably, but would it be possible to make it so that one could easily swap the hash function? Like I am curious about the details here. I think it would be. Clients probably will have to implement a couple of commonly used hash functions, and so forth. I am not sure how it would work in practice or if it is worth it at all. I am interested in all the details though.
Re:BitTorrent v2 by oconnor663 on Tuesday September 08, 2020 @01:01AM (#24405345)
In addition to loeg's point about timing, I'll add that BLAKE3 has its own internal tree structure. It would be unfortunate to have two tree structures layered on top of one another, both because it's "ugly" and because it won't do as good a job of parallelizing things. However, unifying the tree structures would be a big commitment. Every detail of the layout would need to be exactly as it is in BLAKE3. There wouldn't be any space for custom metadata on interior tree nodes, for example. I'm not familiar with the protocol details of BitTorrent myself, but I wouldn't be surprised if that unified approach turned out to be too limiting. (But for a file/tree project that does use the exact BLAKE3 structure, see github.com/oconnor663/bao.)
Re:BitTorrent v2 by somedude11 on Tuesday September 08, 2020 @04:07AM (#24406224)
SHA2 is hardware accelerated on many new CPUs, Blake family not so much.
Re:BitTorrent v2 by johnisgood on Tuesday September 08, 2020 @12:21PM (#24409907)
I know, but according to the graphs, it is much more faster than SHA, despite hardware acceleration, so I am not sure.
Re:BitTorrent v2 by oconnor663 on Tuesday September 08, 2020 @01:06PM (#24410370)
That depends on the platform, the size of the input, and whether multithreading is used.

On Ice Lake, where BLAKE3 benefits from AVX-512 and SHA-256 benefits from the SHA extensions, BLAKE3 seems to do better on both long and short messages. But maybe surprisingly, SHA-256 does better in a medium-length regime, where SHA-256's poorer startup time* has been mostly amortized out, but BLAKE3's chunk parallelism hasn't yet kicked in. See for example the 1536-byte results here: https://bench.cr.yp.to/results-hash.html#amd64-icelake . Using multithreading would exaggerate BLAKE3's advantage for very long messages (usually about 1 MiB and above), but it wouldn't improve the results for any of the message lengths measured there.

* I don't actually know where SHA-256's startup overhead comes from. Maybe someone who knows more could jump in here?

On ARM chips, the performance benefits of NEON are less dramatic than AVX-512, and the performance advantage of SHA-256 hardware acceleration is comparatively larger. I think it's rare for BLAKE3 to beat accelerated SHA-256 on ARM without at least some multithreading, but I've only personally benchmarked a few Raspberry Pis, and I want to be careful not to overgeneralize.