The Billion Row Challenge (1BRC) – Step-by-Step from 71s to 1.7s

Re:The Billion Row Challenge (1BRC) – Step-by-Step from 71s ... by Keyframe on Thursday February 22, 2024 @04:30PM (#39473543)

I went to the original repo, it's indeed a RAM disk. That makes it clear then. I was already almost excited about some IO wizardry going on.

Results are determined by running the program on a Hetzner AX161 dedicated server (32 core AMD EPYC™ 7502P (Zen2), 128 GB RAM).

Programs are run from a RAM disk (i.o. the IO overhead for loading the file from disk is not relevant), using 8 cores of the machine.

[ Parent | on HN ]

Re:The Billion Row Challenge (1BRC) – Step-by-Step from 71s ... by haxen on Thursday February 22, 2024 @04:38PM (#39473635)

Running without RAM cache would be a great followup to this challenge. I think a time around 2-3 seconds should be achievable. But, it would be highly sensitive to the hardware setup and how well the disk-reading code is placed on cores relative to the connection to the disk. Not sure what it would take to allow hundreds of contestants to benefit from the best arrangement.

[ Parent | on HN ]