Ask HN: Best log aggregation solution for a PHP app
https://news.ycombinator.com/item?id=13539592At work I have been given the task to study log aggregation tooling for an e-commerce platform.
Current situation:
- logs are stored locally on each server, preproduction, staging and production - no log replication whatsoever - no aggregation, no monitoring on errors - volumetry is 'quite high' but I have not been able to stat is precisely yet
Goals:
- monitor application logs, not performance - Self-hosted preferably, open source - Remote log replication - Real-time reporting, ability to "see" production errors quickly - Alerting (Slack, mail...)
I have been studying different solutions but it seems none is absolutely perfect and here are the main pitfalls as I understand them:
Logs > ELK - Logstash is the bottleneck, a cluster of 5 servers seems to be recommended
Logs > MQ > InfluxDB > Grafana - no clustering in free version of InfluxDB
Logs > New Relic (free version) - No data retention (3 days), no deployment tracking
Logs > Syslog > rsyslog > syslog-ng > logz.io
Sentry?
----
I would appreciate any feedback on the matter. Thank you!