News for hackers. Stuff that matters.
Re:The One Billion Row Challenge by dvko on Friday January 05, 2024 @07:56AM (#38878593)
There’s an implementation in C which should run in well under 2 seconds on your M3.

https://github.com/dannyvankooten/1brc

Re:The One Billion Row Challenge by corlinpalmer on Saturday January 06, 2024 @02:07AM (#38889196)
Under 2 seconds seems rather impossible on my machine since the disk maxes out at 5.1 GB/s. This one ran in 7.4s on my M3:

`bin/analyze measurements.txt 30.34s user 16.28s system 629% cpu 7.406 total`

Re:The One Billion Row Challenge by dvko on Saturday January 06, 2024 @03:16AM (#38889573)
On most Linux distributions and when the file is mmap'd, if you run it a second time the data will still be in RAM and not have to be read from disk. This gets the runtime down to 1.5s for this AMD 2950x (https://github.com/gunnarmorling/1brc/discussions/46#discuss...).

With SIMD and certain assumptions about the input this can seemingly be further reduced to well under a second, eg see https://github.com/gunnarmorling/1brc/discussions/138.