Re:Appending to a file from multiple processes
by rpcope1 on Thursday August 04, 2016 @03:01AM
(#12223898)
If you have the ability to (minimally) parse what is coming out of your pipe, and you know that your writes are not interleaved, and you know the size of the record you're writing before you write it, you can prefix each record with the size, and use this on the reader side to fetch only upto the record length (read(2) syscall and friends allow you to specify a maximum number of bytes to fetch), for each record. Does this answer your question?
[ Parent
| on HN ]
Re:Appending to a file from multiple processes
by catern on Friday August 05, 2016 @10:23PM
(#12236748)
I don't believe what you're describing works. You're suggesting reading first the size of a record (stored itself in a fixed-size field), then reading the rest of it?
But unfortunately that is not atomic. Another process could read into the data between your first and second reads.