News for hackers. Stuff that matters.
Re:Security for Elasticsearch is now free by nickserv on Monday May 20, 2019 @05:36PM (#19965498)
I can understand they need to make money, but still a bit shady. Honestly though it's not something critical for my needs. Now if they could lower the resource hogging a bit, that would interest me... Maybe even pay for that.
Re:Security for Elasticsearch is now free by jrockway on Monday May 20, 2019 @05:51PM (#19965643)
I use ELK for Kubernetes and network device logs, and I'm very much with you -- full text search is great, but it sure can be slow, even when running on $1000/month of AWS hardware.

The conclusion that I've reached is that the whole lucene model for logs is kind of outdated. Why am I tuning Java GC params to run "grep foo /logs". I think computers today can do fine with sharded flat files, a minimal index ("which node contains logs from pod foo-2387438-2384738 at 12:34AM"), and then just scale horizontally over (log messages, searches).

I hope my friends over at Tailscale are doing that and I can just move off ES entirely ;)

Re:Security for Elasticsearch is now free by ReidZB on Monday May 20, 2019 @11:25PM (#19967369)
I believe Loki [1] is intended to basically run "grep foo" at scale (plus some extra niceties like labels). I haven't used it, but it seems interesting.

[1]: https://grafana.com/loki

Re:Security for Elasticsearch is now free by vorpalhex on Monday May 20, 2019 @10:31PM (#19967157)
ELK stack user here - we actually found logstash to be our bottleneck. Changing it out for fluentd fixed our woes.
Re:Security for Elasticsearch is now free by nickserv on Tuesday May 21, 2019 @02:45AM (#19968034)
Same here, fluentd is much better, performance wise.

But then I had to give ES more RAM because it couldn't take the hammering.

In fact, increasing the throughput to ES was causing some pretty spectacular crashes, with the /var/log partition at 100% because of the verbosity of the dumps.

Re:Security for Elasticsearch is now free by majkinetor on Tuesday May 21, 2019 @05:51AM (#19968762)
Logstash sucks from both operational and developing perspective. I replaced it everywhere I could by sending structured logs directly from the app or by using newer integrated beats features.
Re:Security for Elasticsearch is now free by ryanworl on Monday May 20, 2019 @08:09PM (#19966548)
Is Tailscale building a logging product?