News for hackers. Stuff that matters.
Re:Show HN: Log.io (Realtime log monitoring, powered by node... by viraptor on Sunday June 05, 2011 @06:48PM (#2623018)
> Log.io has no persistence layer. Harvesters are informed of file changes via inotify, and log messages hop from harvester to server to web client via socket.io.

This struck me as really odd. Why is it done that way? Most stuff is logged via syslog. Those things are already in the buffer and it seems the local nodes are doubling the seek+read of what was just written. Downsides:

- double disk action for every log line

- inotify overhead for the system

- logging requires local disk space

- disk io delays mean collection delays

- when disk get full / fail on write you don't collect the message about the failure if one was attempted to be written

- when syslog daemon decides that messages are coming in too fast, it might drop them from the for_the_disk buffers - this way the only copy was lost (not 100% sure, but I believe syslog-ng can send to one destination even if another is blocked)

I really can't see positive sides of this approach. Logstash seems to handle everything log.io does and much more... am I missing something?

Re:Show HN: Log.io (Realtime log monitoring, powered by node... by guan on Sunday June 05, 2011 @07:41PM (#2623133)
A lot of things don’t use syslog. For example, default configurations of Apache write logs directly without syslog. The benefit of this method is that it is general enough to work for most people without configuration changes, and I assume they could always add syslog support later.
Re:Show HN: Log.io (Realtime log monitoring, powered by node... by viraptor on Sunday June 05, 2011 @08:07PM (#2623187)
Log.io assumes multiple nodes. If you want any kind of visit statistics in that scenario, you're already collecting the logs somehow. Unless you rolled out some homemade script for that purpose, you're probably using remote syslog anyways.

Or do you store your logs on a random server that request happened to hit / session was bound to?