Oh, and: not to speak of that one person somewhere in the chain of command who keeps barking out orders to randomly disable Splunk ingestion for individual instances due to cost.
Centralized logging gives me headaches and pain.
It might be useful to add a few tips for how to tail better, like using `journalctl -f` or `less +F` or ways to tail multiple files.
In Splunk, something like "sourcetype=your_sourcetype | timechart count by status" gives you a chart of HTTPS statuses (200 vs 404 vs 503) over time. After learning that the language it becomes intuitive and is a really really powerful when trying to analyze logs because a visual graph is just so much better than tailing log files. If your centralized logging tool doesn't give you same access to graphing as Splunk does, I'm sorry but it's not good enough to replace 'tail'. Splunk really is the state of the art here and I've yet to see anything that's competitive. (But tbh, I've not looked very hard - the only downsize with Splunk is its cost once you get over a certain size.)
In particular, Splunk's query language makes it easy (once you figure it out) to build graphs, so a pie chart or bar graph of, say, how many requests across the whole fleet are hitting a particular endpoint.