ELK is a very standard stack for collecting and disseminating logs. It's a fairly natural, and i think sensible, step to build a pattern-based alerting tool as a bag on the side of ELK.

If you were going to do it more properly, the first choice would probably be Riemann, which has already seen some adoption in this role. There are already plugins on both sides to forward events to the other:

https://www.elastic.co/guide/en/logstash/current/plugins-out...

http://riemann.io/api/riemann.logstash.html

With all respect to the very smart author of Riemann, as a practical matter, its use of Clojure is simply not a hurdle most SRE types are going to overcome to use it.

ElasticSearch is a great search engine. It's not a CEP or SEP. I don't mean to discourage its use, but for this purpose it's not a good fit. It does make sense to tee incoming logs to both as they suit different purposes, but I'd delegate pattern analysis to the latter.

Using Clojure, especially to slap together Riemann configs, is really not an insurmountable obstacle, especially when something as useful as Riemann is on the line. The more the self-defeating attitude that "Clojure's too hard" propagates the less likely people will be to even try. Most SRE's are pretty smart, I believe in them.

There are some great commercial options in this space like SignalFX and Sumo Logic. The cost of Riemann's learning and implementation challenges would have to be made significantly less for entire teams to make it a viable alternative to those, in my experience.

If anyone runs a team of 25 or more SREs and effectively uses Riemann (and there's more than 3 domain experts), and is not otherwise a Clojure shop, I'd love to hear from you.

> as a practical matter, its use of Clojure is simply not a hurdle most SRE types are going to overcome to use it

And yet, it's the most widely used option in this role. Crazy days.

Can you site something specific for this claim? Are you talking about clojure or riemann or both?

SEP? I know CEP. Riemann is great but I think you are right, there's a lot of people who aren't willing to pick up some clojure for whatever reason.