Distributed Logging Architecture in the Container Era

Re:Distributed Logging Architecture in the Container Era by ktamura on Friday August 12, 2016 @01:18PM (#12277241)

>In particular, the ability to live-stream ("tail") logs seems to be a feature generally missing from logging aggregators.

If you are talking about tailing log files live, Fluentd has supported it from Day 1: http://docs.fluentd.org/articles/in_tail

Also, as other sibling comments mention, there are tools, both SaaS and open source, that you can use as a destination of the logs Fluentd tails/listens/collects.

* Elasticsearch: https://www.digitalocean.com/community/tutorials/elasticsear...

* Graylog: http://www.fluentd.org/guides/recipes/graylog2

* Scalyr: https://github.com/scalyr/scalyr-fluentd

* Loggly: https://www.loggly.com/blog/stream-filtering-loggly-fluentd/

* SumoLogic: https://gist.github.com/d-smith/8d3e7d53db772c6a7845

* Papertrail: https://github.com/docebo/fluent-plugin-remote-syslog

(and literally hundreds of others)

>though I rather wished Heka had taken off; it's much more flexible and in theory leaner and faster since it's Go

Heka was a great project, and a drop-in binary (as opposed to requiring a VM like Ruby) approach was interesting if not compelling in certain situations. That said, I never saw any benchmark that showed Heka was materially faster than Fluentd (or Logstash, for that matter). A lot of speed in this type of complex software comes from data structures/algorithms, an appropriate use of low-level language bindings, etc.

While language plays a role in the speed of software, it's hardly the only factor. As you said, it's only in theory, not in practice =)

[ Parent | on HN ]

Re:Distributed Logging Architecture in the Container Era by lobster_johnson on Friday August 12, 2016 @03:46PM (#12278256)

I meant tailing the aggregated logs themselves. I have looked at the CLI tools that exist for some of those SaaS services, and I have not found anything half decent.

For example, you will definitely want to filter on labels (including regexps) while tailing, and such filtrering should support adjustable context (both # of lines and time interval) and should support an optional time range to scroll back into history ("tail from 2pm"). And of course, grepping of historical entries.

Another thing I am not impressed with is pricing. Loggly seems the most reasonable in terms of price per volume, except it limits the number of "team members" to a ridiculous degree (5 users or something like that).

I set up Graylog once and wasn't impressed with it. Its reliance on Elasticsearch means it is quite static when it comes to the schema/input format. You can't change the settings; there is no reindexing support (or at least this was the case when I tried it, a year ago or so). Also, don't think it has any CLI tools?

[ Parent | on HN ]