News for hackers. Stuff that matters.
Re:Confluent, a company for Apache Kafka and realtime data by lrm242 on Friday November 07, 2014 @09:16AM (#8572541)
autossh + ssh -C + tail -F ?

kafka sounds like an enormous overkill. If you want to store the logs locally while tailing, just add in a tee.

Re:Confluent, a company for Apache Kafka and realtime data by beagle3 on Friday November 07, 2014 @09:28AM (#8572597)
Both this solution (and easytiger's above) work if disconnects happen rarely and for short times.

But it my case, I have "30 minutes on, 5 minutes off, 90 minutes on, 90 minutes off, 2 minutes on" kind of situations, in which anything that doesn't track what was already transferred and what wasn't, will lose data. (zeromq's buffers also have limited capacity and/or are tied to a process on the other side - if it restarts, buffers are gone).