Using mmap is basic systems programming knowledge and not "incredible" even though it's a useful contribution.

Glad to see we're slowly coming to the realization that wrangling NumPy and Pandas doesn't actually count as Software Engineering.

Let's not gatekeep what Software Engineering is or isn't.

it's as much engineering as is gluing mmap, recv and send... there are people who can do that and can't piece together a dataframe pipeline, no need to be passive agressive here

> "it's as much engineering as is gluing mmap, recv and send"

As you can see from comments up this thread (ref. various GitHub issues) - the people "gluing mmap" don't actually have a single clue. They can't properly measure memory consumption (they don't understand what the numbers they're seeing actually mean). They don't understand how paging, swapping or virtual memory work. They don't actually understand the concept of memory-mapped files, why they're there and how they work. They can't explain why their code behaves differently when using memory-mapped files.

Moments like this are here to remind you that there's actual knowledge and skill to building scalable and efficient software, and that hustling and copy-pasting StackOverflow examples will only get you so-far, as will "piecing together dataframe pipelines" in Python.

…I don’t see how this is related?

Systems Programmer is itself a niche field, when compares to other fields of interest in Software Developement. So let's not trivialize anything, cause it can be amazing to someone, but not to another. It depends from person to person.

I think reducing this to "using mmap is basic" is pretty unfair.

The trick she did overriding malloc & friends to validate that the optimization would be worth doing is, in my mind, one of the high-points of the paper. It's a very clever way of making a meaningful measurement, which was the keystone of the entire change.

I've never heard of, thought of, or used that trick, and the fact she had it in her arsenal to apply to this very specific situation is pretty impressive, to me at least.

She is very good indeed, but this mmap thing is not comparable to her previous work.

Hopefully we should publicize this kind of achievement as a way to teach more devs about mmap... (this really should be common knowledge)

Of course its simpler, I just wanted it to make sure the comment still related to her work on this :) In summation, what she’s done is awesome.

I suspect you're trolling but Jart is not a genius for co-authoring a 300 line change to use mmap(), this is a pretty typical system engineering optimization that goes on in big companies, which is usually framed as a performance improvement for certain use cases with pros and cons, not "Hacker news front page 100X revolutionary performance improvement".

holy cow, her website is enough to give me imposter syndrome https://justine.lol/