Re:Prefix sum: 20 GB/s (2.6x baseline)
by ashtonsix on Tuesday October 14, 2025 @06:44PM
(#45585930)
Oh right. That's sensible enough. Makes total sense to parallelise across multiple cores.
I wouldn't expect a strictly linear speed-up due to contention on the memory bus, but it's not as bad as flat-lining after engaging 2-3 cores. On most AWS Graviton instances you should be able to pull ~5.5 GB/s per-core even with all cores active, and that becomes less of a bottleneck when considering you'll typically run a sequence of compute operations between memory round-trips (not just delta).
Even if one is willing to adopt such an encoding scheme, you'd still want to optimize what you have here anyways though. It also doesn't help, as mentioned, if the goal is actually latency of small streams rather than throughput of large ones.