Re:Linux's fsync() woes are getting some attention
by deathanatos on Tuesday March 11, 2014 @01:30AM
(#7377611)
> So a little bit of data loss is okay, but a lot isn't?
I don't think that's the issue being put forth here, all the way to the most-ancestor comment. It's not between a bit of data loss and a lot of data loss: if the power fails, I fully expect that some data that was in buffer might not have made it to the disk. But that's okay: I left my file in a state that's still valid (using a journal or something), and I can recover from it.
But if write B happens before write A, my file is corrupt, and it's game over. I don't need to override the kernel's I/O scheduler to say "write this write right now!", I just need to ensure that A goes before B. That's it; it can happen whenever, but it must happen in that order.
Currently, the only solution seems to be fsync and forcing everything to disk right now, and screwing over any buffering what-so-ever¹. I just want to specify to the kernel, "here's my data, here's how it must be written, but you can buffer and write it when convenient, as you have a better picture of the whole system I/O."
¹It's funny that the article mentions Postgres/MySQL, as I've heard of fsync woes through browsers using SQLite.
I don't think that's the issue being put forth here, all the way to the most-ancestor comment. It's not between a bit of data loss and a lot of data loss: if the power fails, I fully expect that some data that was in buffer might not have made it to the disk. But that's okay: I left my file in a state that's still valid (using a journal or something), and I can recover from it.
But if write B happens before write A, my file is corrupt, and it's game over. I don't need to override the kernel's I/O scheduler to say "write this write right now!", I just need to ensure that A goes before B. That's it; it can happen whenever, but it must happen in that order.
Currently, the only solution seems to be fsync and forcing everything to disk right now, and screwing over any buffering what-so-ever¹. I just want to specify to the kernel, "here's my data, here's how it must be written, but you can buffer and write it when convenient, as you have a better picture of the whole system I/O."
¹It's funny that the article mentions Postgres/MySQL, as I've heard of fsync woes through browsers using SQLite.