News for hackers. Stuff that matters.
Re:Appending to a file from multiple processes by ww520 on Thursday August 04, 2016 @12:38AM (#12223417)
Something like syslogd?
Re:Appending to a file from multiple processes by DougWebb on Thursday August 04, 2016 @09:25AM (#12225299)
Similar, but different requirements. All of the messages were grouped by sending process, and when the sending process said its session was over its messages were fed into a post-processing queue. There was support for holding messages in memory or on disk, and data-preserving fallbacks if the post-processing halted or got too backed up. The message rate varied quite a bit; during the US business day the rate was at least doubled. We used that to give the post-processing queue time to catch up off-hours.

This was designed about 16 years ago, so well before the big-data tools, ssds, or powerful machines we have today.