News for hackers. Stuff that matters.

Ask HN: Best log aggregation solution for a PHP app

At work I have been given the task to study log aggregation tooling for an e-commerce platform.

Current situation:

- logs are stored locally on each server, preproduction, staging and production - no log replication whatsoever - no aggregation, no monitoring on errors - volumetry is 'quite high' but I have not been able to stat is precisely yet

Goals:

- monitor application logs, not performance - Self-hosted preferably, open source - Remote log replication - Real-time reporting, ability to "see" production errors quickly - Alerting (Slack, mail...)

I have been studying different solutions but it seems none is absolutely perfect and here are the main pitfalls as I understand them:

Logs > ELK - Logstash is the bottleneck, a cluster of 5 servers seems to be recommended

Logs > MQ > InfluxDB > Grafana - no clustering in free version of InfluxDB

Logs > New Relic (free version) - No data retention (3 days), no deployment tracking

Logs > Syslog > rsyslog > syslog-ng > logz.io

Sentry?

----

I would appreciate any feedback on the matter. Thank you!

( 1 comment | 2 points | on HN )
Re:Ask HN: Best log aggregation solution for a PHP app by stympy on Wednesday February 01, 2017 @07:50AM (#13540272)
Since you have nothing in place at the moment, I recommend you get something in place while you are evaluating your options.

At honeybadger.io we use papertrail for log monitoring, and we love it, but there are plenty of hosted options. Set up a free trial and you'll at least get aggregation and monitoring now, plus an idea of what kind of logging volume you'll need to support with your self-hosted solution.