Re:Redis latency spikes and the Linux kernel: a few more det...
by seiji on Monday November 03, 2014 @01:45PM
(#8552602)
See the "How it works" section under "Snapshotting" at http://redis.io/topics/persistence
[ Parent
| on HN ]
Re:Redis latency spikes and the Linux kernel: a few more det...
by Ono-Sendai on Monday November 03, 2014 @02:03PM
(#8552711)
Ok.
A couple of points - you can do this kind of persistence with an immutable data structure without forking.
Secondly, huge pages help to reduce memory latency for large data sets quite significantly.
Re:Redis latency spikes and the Linux kernel: a few more det...
by seiji on Monday November 03, 2014 @02:09PM
(#8552749)
you can do this kind of persistence with an immutable data structure without forking
How?
huge pages help to reduce memory latency
Yup, but this is about _transparent_ huge pages which break databases all over the place: https://blogs.oracle.com/linux/entry/performance_issues_with... and http://dev.nuodb.com/techblog/linux-transparent-huge-pages-j... and http://scn.sap.com/people/markmumy/blog/2014/05/22/sap-iq-an... and http://www.percona.com/blog/2014/07/23/why-tokudb-hates-tran... and (a dozen more including varnish, mongo, hadoop, ...)
Re:Redis latency spikes and the Linux kernel: a few more det...
by Ono-Sendai on Monday November 03, 2014 @02:34PM
(#8552915)
If you have a pointer to an immutable data structure, you know it won't change, so you can write it out to disk at your leisure.
Re:Redis latency spikes and the Linux kernel: a few more det...
by justincormack on Monday November 03, 2014 @03:26PM
(#8553256)
But the immutable data structure is not that much different from the copy-on-write pages the kernel gives you, likely to involve the same amount of copying. And disk IO is blocking so you need at least a thread, so it is a natural strategy.