Re:The One Billion Row Challenge
by anonymoushn on Thursday January 04, 2024 @07:41AM
(#38866269)
It seems like you have to run the slow collision-resistant thing over the whole file.
[ Parent
| on HN ]
Re:The One Billion Row Challenge
by londons_explore on Thursday January 04, 2024 @08:25AM
(#38866701)
Collision detection is far cheaper. One method:
Simply add up all the bytes of all seen placenames while running your fast algorithm. This is effectively a checksum of all bytes of placename data.
Then at the end, calculate the 'correct' name-sum (which can be done cheaply). If it doesn't match, a collision occurred.
Re:The One Billion Row Challenge
by anonymoushn on Thursday January 04, 2024 @08:34AM
(#38866775)
You can design inputs that defeat this scheme though. I thought the point was to be correct for all valid inputs.