News for hackers. Stuff that matters.
Re:Redis latency spikes and the Linux kernel: a few more det... by antirez on Monday November 03, 2014 @12:13PM (#8551959)
Can't test it right now, but seems smart, to check if they got split into smaller pages.
Re:Redis latency spikes and the Linux kernel: a few more det... by nteon on Monday November 03, 2014 @01:17PM (#8552420)
I just tested - while running the redis-benchmark command you listed after a BGSAVE smaps lists all anonymous memory as AnonHugePages. So it ends up as THP, although I suppose its possible that it is broken into 4k pages and then adjacent 4k pages are merged back into huge pages. Will see if I can figure out how the lazy PTE copying works.
Re:Redis latency spikes and the Linux kernel: a few more det... by nteon on Monday November 03, 2014 @03:03PM (#8553102)
as a followup, I don't see large latency with huge pages, but I have kernel samepage merging disabled.

I would be interested if this also decreases latency for you:

echo 2 > /sys/kernel/mm/ksm/run

Also check stats here: /sys/kernel/mm/ksm/*

And some documentation here: https://www.kernel.org/doc/Documentation/vm/ksm.txt

Re:Redis latency spikes and the Linux kernel: a few more det... by antirez on Monday November 03, 2014 @06:05PM (#8554164)
Strange... you should definitely see it if you are using jemalloc now that it's clear it's about mass-COW.
Re:Redis latency spikes and the Linux kernel: a few more det... by nteon on Monday November 03, 2014 @10:55PM (#8555076)
I do see it now... wasn't testing right before. Goes away as you say when disabling transparent THP. KSM has no effect, contrary to what I thought earlier.
Re:Redis latency spikes and the Linux kernel: a few more det... by antirez on Tuesday November 04, 2014 @03:50AM (#8555631)
Thanks a lot for independent confirmation of the issue.