Until you need them, the you'd trade anything to get them. Logs are like backups, you don't need them most of the times, but when you need them, you really need them.
On the flip side, the tendency is to over-log "just in case". A good compromise is to allocate a per-project storage budget for logs with log expiration, and let the ones close to the coal-face figure out how they use their allocation.
Even at very immature organizations, log data within a service is usually structured.
Even in my personal projects if I'm doing anything parallel structured logging is the first helper function I write. I don't think I'm unrepresentative here.
> Even at very immature organizations, log data within a service is usually structured.
unless the framework provides it by default, I've never seen this actually happen in real life. Sure I've seen a lot of custom telegraf configs, status end points and the like, but never actual working structured logging.
When I have seen structure logs, each team did it differently, The "ontology" was different (protip: if you're ever discussing ontology in logging then you might as well scream and run away.)
actually, how many of your messages include the time and date, and how many different ways of displaying timestamp exist in those messages?
That is why I say logs are unstructured, because all but a very few places actually have the discipline to enforce a single log structure.
Metics tell you where and when something when wrong. Logs tell you why.
However, a logging framework, which is generally lossy, and has the lowest level of priority in terms of deliverability is not an audit mechanism. especially as nowhere are ACLs or verifiability is mentioned. How do they prove that those logs originates from that machine?
If you're going to have an audit mechanism, some generic logging framework is almost certainly a bad fit.