News for hackers. Stuff that matters.
Re:A distributed and coordination-free log management system by packetized on Tuesday January 17, 2017 @05:04PM (#13421843)
While super-interesting (and something that I wish I had time to really evaluate against Elasticsearch/Solr), I feel like it's missing one of the most useful things about Loggly/ES/Solr - the ability to quickly visualize log trends. And despite the statement that it's a "distributed and coördination-free log management system for big ol' clusters", I'm confused why you would want a "big ol' cluster" without visualization or field searches more complex (and less computationally expensive) than regexes. Am I missing something?

e: To be fair, I'm not attempting at all to defend Elasticsearch/Solr, just slightly confused about the actual use cases in prod.

Re:A distributed and coordination-free log management system by sagichmal on Tuesday January 17, 2017 @05:07PM (#13421869)
What does "the ability to quickly visualize log trends" mean concretely?
Re:A distributed and coordination-free log management system by Juliate on Wednesday January 18, 2017 @12:08PM (#13427995)
There are trends over the long run, and there are trends in the short span.

Something like glTail (http://www.fudgie.org/ ) or even with a sonification, those would be nice complements, if well tuned, to give some sort of a sense of what is happening.

I found those were not many. Is that there is actually little benefit to use these sort of vis/sonification, or that there's just not many projects that tried to scratch the surface in this area?

Re:A distributed and coordination-free log management system by packetized on Tuesday January 17, 2017 @05:16PM (#13421970)
In my mind, some sort of graphical user interface that would allow you to query data and have the log system return results that are easily parsed. At the very least, have some sort of API that allows you to do this. The /query endpoint exists, but doesn't seem to return anything that's really fundamentally useful from a large-scale perspective.
Re:A distributed and coordination-free log management system by benley on Tuesday January 17, 2017 @08:53PM (#13423376)
I agree that the nice graphical tools available for Elasticsearch are super useful, but it seems to me that those are not a feature of the logs storage or query interface, but rather one of an external app (e.g. kibana) that queries the system.

It would actually be somewhat concerning to me if a system like Oklog had all those graphing and visualization features built-in; that would be quite some scope creep for a distributed storage engine :-P

Re:A distributed and coordination-free log management system by dstroot on Tuesday January 17, 2017 @09:32PM (#13423505)
TL;DR

So, can I run Kibana against OKlog?

Totally agree on the separation of concerns: visualization vs storage and management.

Re:A distributed and coordination-free log management system by NikolaeVarius on Tuesday January 17, 2017 @09:35PM (#13423521)
I disagree. Elasticsearch itself is doing most of the heavy lifting. Kibana is developed in sync with Elasticsearch so that it always has the ability to use most of the features of Elasticsearch.

The nice visualization methods are a direct result of the aggregation abilities of Elasticsearch

Re:A distributed and coordination-free log management system by grey-area on Wednesday January 18, 2017 @12:36AM (#13424276)
Ways to query the data are important and on this newly launched pkg probably need some work (haven't looked at that side of it), but visualisation does not belong in a logging tool, it belongs in external tools querying that tool.
Re:A distributed and coordination-free log management system by xemoka on Tuesday January 17, 2017 @10:55PM (#13423836)
Could it be used in combination with something like GoAccess.io?