Re:Logging can be tricky
by jacques_chester on Tuesday September 23, 2014 @10:40PM
(#8359629)
Have you experimented with a log-structured filesystem? I once saw similar high-variation behaviour in a logging problem on extfs. Switching to NILFS pretty much got rid of variable latency entirely.
Re:Logging can be tricky
by evan_miller on Tuesday September 23, 2014 @10:52PM
(#8359665)
Not recently, no. I do know that a journaled filesystem can exacerbate this sort of problem as it can make extra work. For example: http://lwn.net/Articles/328363/
In a few cases in the past when dealing with unimportant data I have downgraded to ext2 for a nice performance bump.
Re:Logging can be tricky
by jacques_chester on Wednesday September 24, 2014 @07:31AM
(#8360846)
Just to make sure we're talking about the same thing, a journalling FS isn't the same thing as a log-structured FS.
The first has a write-ahead log, the latter is basically just a log. So immediately writing to disk is relatively simple.
Re:Logging can be tricky
by evan_miller on Wednesday September 24, 2014 @08:56AM
(#8361150)
Yes. I haven't done anything with a log-structured FS. I have only played around with conventional filesystems.
The trouble outlined in my blog post is that the logging framework was calling fsync() -- that is, specifically asking to flush the page cache all the way to disk.