News for hackers. Stuff that matters.
Re:Logging can be tricky by evan_miller on Tuesday September 23, 2014 @10:29PM (#8359597)
With fsync() taken out of the equation this is essentially how a naive logger operates. The linux page cache plays the role of shared memory and only under rather heavy contention will a write() incur latency.

The trouble outlined in my blog post is that the logging framework was calling fsync() -- that is, specifically asking to flush the page cache all the way to disk.

Re:Logging can be tricky by jacques_chester on Tuesday September 23, 2014 @10:40PM (#8359629)
Have you experimented with a log-structured filesystem? I once saw similar high-variation behaviour in a logging problem on extfs. Switching to NILFS pretty much got rid of variable latency entirely.
Re:Logging can be tricky by evan_miller on Tuesday September 23, 2014 @10:52PM (#8359665)
Not recently, no. I do know that a journaled filesystem can exacerbate this sort of problem as it can make extra work. For example: http://lwn.net/Articles/328363/

In a few cases in the past when dealing with unimportant data I have downgraded to ext2 for a nice performance bump.

Re:Logging can be tricky by jacques_chester on Wednesday September 24, 2014 @07:31AM (#8360846)
Just to make sure we're talking about the same thing, a journalling FS isn't the same thing as a log-structured FS.

The first has a write-ahead log, the latter is basically just a log. So immediately writing to disk is relatively simple.

Re:Logging can be tricky by evan_miller on Wednesday September 24, 2014 @08:56AM (#8361150)
Yes. I haven't done anything with a log-structured FS. I have only played around with conventional filesystems.