News for hackers. Stuff that matters.

A comment on Appending to a file from multiple processes. Parent comment

Re:Appending to a file from multiple processes by altano on Wednesday August 03, 2016 @06:00PM (#12221899)
The issue is the contention caused by all the processes waiting on file IO behind the lock, not the locking
[ Parent | on HN ]
Re:Appending to a file from multiple processes by caf on Wednesday August 03, 2016 @11:24PM (#12223175)
You don't actually need to hold the lock while doing I/O - just while allocating yourself a region of the file that's yours to exclusively write to (eg. by opening without O_APPEND and instead using pwrite(2) to write at a specific offset).
[ Parent | on HN ]
Re:Appending to a file from multiple processes by danbruc on Wednesday August 03, 2016 @06:10PM (#12221955)
If I/O were the bottleneck then that would be the case with or without a lock, wouldn't it?
[ Parent | on HN ]
Re:Appending to a file from multiple processes by altano on Thursday August 04, 2016 @02:26AM (#12223774)
It depends on what has to wait for the IO to complete. ETW, for example, is a Windows-wide event framework and it would be untenable for everything using it to wait on the file IO of everything else making use of it. So, it has a solution that makes use of locks but nothing ever has to wait on file IO.

File IO is always expensive, the trick is always in how you work around that.

[ Parent | on HN ]