News for hackers. Stuff that matters.

A comment on What about K?. Parent comment

Re:What about K? by orlp on Monday February 10, 2025 @06:50PM (#43006870)
Yes, I did already propose (at the office) a parity-agnostic chunker (we only need the number of lines + a splitpoint from the chunker) that can do parallel work and only needs a small moment of synchronization to find out which of the two parities it is to lock in a final answer. There would still be a global serial dependency, but on blocks rather than on bytes.

But we only have a finite amount of time and tons and tons of work, so no one has gotten around to it yet. At least now we know that it might be worthwhile for >= ~32 core machines. PRs welcome :)

[ Parent | on HN ]
Re:What about K? by mlochbaum on Monday February 10, 2025 @07:06PM (#43007037)
All right, just threw me off a little that you'd consider speculating or backwards decoding as I wouldn't expect them to be easier, or significantly faster (or maybe you consider parity-independence to be speculation? I can see it).
[ Parent | on HN ]
Re:What about K? by orlp on Monday February 10, 2025 @08:00PM (#43007604)
Yes, I meant parity-independence with speculation. Essentially you assume either you are or are not within a string at the start and do your computation based on that assumption, then throw away the result with the unsound assumption. Both assumptions can share most of their computation I believe, so I can understand one might see it from the other perspective where you'd start with calling it parity-independence rather than speculation with shared computation.
[ Parent | on HN ]