Re:A collection of lock-free data structures written in stan...
by ot on Wednesday May 10, 2023 @07:16PM
(#35894854)
This seems to me like a parallel accumulation problem, why not have each thread accumulate a filter on a subset of the data (so no locking involved), and then reduce the results (which is just an OR of all the local accumulations)?
[ Parent
| on HN ]
Re:A collection of lock-free data structures written in stan...
by sakras on Wednesday May 10, 2023 @09:23PM
(#35896067)
Parallel reductions are more heavy-weight synchronizations than locks. Say we have 64 partitions, then we need to perform 6 levels of tree reduction, or avoid parallelism completely and perform the reduction on a single thread. Either way it was slower.
The locking strategy very rarely had any reduction in parallelism due to the randomized lock-taking.
There were also other reasons, such as not wanting to replicate the filter per-thread.