News for hackers. Stuff that matters.
Re:The One Billion Row Challenge by usefulcat on Thursday January 04, 2024 @10:59AM (#38868555)
> IO will dominate the running time for this, and JSON parsing will be second.

Memory bandwidth might dominate, but probably not I/O. The input file is ~12GB, the machine being used for the test has 32GB, and the fastest and slowest of five runs are discarded. The slowest run will usually be the first run (if the file is not already cached in memory), after which there should be little or no file I/O.

Re:The One Billion Row Challenge by nojvek on Sunday January 07, 2024 @05:45AM (#38900115)
Is there a way to validate from app whether all file pages are cached in memory?

What if the code was run against constraints such as Max memory limit in a docker container.