Optimizing Linux Memory Management for Low-latency, High-throughput Databases

Re:Optimizing Linux Memory Management for Low-latency, High-... by caf on Tuesday October 08, 2013 @07:43PM (#6518608)

In regard to conclusion 2, there is another approach here - when you're finished with an old segment, posix_fadvise(..., POSIX_FADV_DONTNEED) can be used to drop it from the page cache.

[ Parent | on HN ]

Re:Optimizing Linux Memory Management for Low-latency, High-... by apurvamehta on Tuesday October 08, 2013 @10:31PM (#6519120)

That would be true if we were using C++. Unfortunately, all our code is in Scala and we use Java NIO libraries to memory map our files. AFAIK, they don't give us the option on using these POSIX calls.

[ Parent | on HN ]

Re:Optimizing Linux Memory Management for Low-latency, High-... by pquerna on Wednesday October 09, 2013 @12:31AM (#6519404)

Cassandra binds to posix_fadvise to do exactly this when writing out new SSTables:

https://github.com/apache/cassandra/blob/trunk/src/java/org/...

[ Parent | on HN ]

Re:Optimizing Linux Memory Management for Low-latency, High-... by apurvamehta on Wednesday October 09, 2013 @12:46AM (#6519448)

Wow.. that's great to know. We will definitely investigate this approach. Thanks for sharing! :)

[ Parent | on HN ]