News for hackers. Stuff that matters.
Re:Loading CSV File at the Speed Limit of the NVMe Storage by wtallis on Saturday October 10, 2020 @03:10AM (#24737154)
> mmap absolutely copies the file contents into the kernel file system cache which is a buffer

Isn't this a bit misleading? mmaping a file doesn't cause the kernel to start loading the whole thing into RAM, it just sets things up for the kernel to later transparently load pages of it on demand, possibly with some prefetching.

Re:Loading CSV File at the Speed Limit of the NVMe Storage by shawnz on Saturday October 10, 2020 @03:13AM (#24737165)
The GP's comment on the other hand seemed to imply it was completely unbuffered.
Re:Loading CSV File at the Speed Limit of the NVMe Storage by wtallis on Saturday October 10, 2020 @03:27AM (#24737207)
"Completely unbuffered" is almost unattainable in practice, so I'm not sure that's a reasonable inference. About the best you can do in general is not do any buffering yourself, and usually explicitly bypass whatever buffering is going on at the next level of abstraction down. Ensuring you've cut out all buffering in the entire IO stack takes real effort.
Re:Loading CSV File at the Speed Limit of the NVMe Storage by monocasa on Saturday October 10, 2020 @03:32AM (#24737225)
> "Completely unbuffered" is almost unattainable in practice, so I'm not sure that's a reasonable inference.

While absolutely true, I've found that fact to be very surprising to a lot of engineers.