IMO the author buried the lede:

> Finally, understand that logging is expensive. I’ve seen entire teams of absolutely brilliant engineers spend years building, managing, and evolving logging infrastructure. It’s a hard problem, made much harder by overburdening the pipelines with unnecessary load.

It's ultimately a piece about engineering cost versus benefit. There's a real cost associated with logging every event with identity, to create an audit trail. Maybe that's necessary in your domain, for compliance reasons or otherwise. In which case it falls into the bucket of machine-read structured logging, and you design for it. But for lots of domains, it isn't strictly necessary.

In which light, allow me to rephrase the argument:

Log as much as you can afford.

Given that the cost of logging is low and continuously falling, the default decision should be "log everything".

"Log only actionable messages" is analogous to advice like "always use bit masks instead of booleans", or only "use two digits to encode years". It assumes algo-economics that are no longer the normal case.

IMO it's also worth considering the issue of synchronous vs asynchronous logging during the process.

Throughput gets better but latency's a tougher nut to crack.

> Given that the cost of logging is low and continuously falling

Everything gets cheaper in time, but the cost of logging is actually quite high.

I don't know why you're getting downvoted, because in my mind you're right in some circumstances. More than once, I've found logging-related performance issues (i.e. logging is near the top of a profiling-driven performance optimization task).

One immediate example that comes to mind was code that had a bunch of logging statements like log.debug(boost::format("%d %x etc", param, param)). Normally the code would run with debug logging turned off, but there's a pretty brutal problem there! The formatted string is always going to be processed, which the logger will immediately discard. Turns out one of these formatted strings actually took a fair bit of processing to generate and was hurting the system.

That's a solved problem. Your logger should be capable of formatting on it's own, and take the arguments to the format string as variadic arguments. This is how Java's ubiquitous SLF4J interface is defined. If the log level isn't low enough, then the arguments are tossed but never composited.

And it's even better with languages with macros, like Rust, Scala, even C++. There won't be any function call to begin with if the corresponding logging level is not enabled, only a branch.

I replaced it with a macro!

The cost of logging is actually quite low relative to the enormous cost of engineering hours spent debugging a system that's flying blind.

I don't know who downvoted you, though I still disagree.

If I was writing an embedded system, or a system with soft-to-hard realtime requirements, or one where flat straight line performance was the only requirement that counted, sure, I might ditch or constrain logging.

These systems might be common by manufactured volume. But most programmers will never touch one.

You're probably thinking of I/O costs. Again, these aren't as a brutal as they used to be -- SSDs swallow bursty writes with aplomb and when those aren't enough, log-structured filesystems are good at smoothing out writes.

As for myself, in the work I do in my day job, I print out to STDOUT/STDERR and let the platform wick those lines away to a central firehose.

   > If I was writing an embedded system, or a system 
   > with soft-to-hard realtime requirements, or one 
   > where flat straight line performance was the only 
   > requirement that counted, sure, I might ditch or
   > constrain logging.

I've worked on embedded systems, and logging is still extremely important, if not more important because of the difficulty of seeing what's going on (it's hard to reproduce events that only happen in a truck while you're inside your office).

In one case, a team had been trying for months to debug a problem, but when proper logging was added, the issue was easily fixed.

> in the work I do in my day job, I print out to STDOUT/STDERR and let the platform wick those lines away to a central firehose.

I think this is the correct approach, but (to the nearest approximation) ~every organization I've worked at that does this has hit saturation limits of "the platform" in extremely short order. Especially if you're doing audit-trail-style logging, in my experience, this becomes the primary bottleneck of your infrastructure very quickly.

Did you come up with particular tactics? Any insights on platform bottlenecks?

I am curious about this. Can you give specifics about the costs of logging in this context?

Not the parent, but I know our production web servers have very limited logging since they were already highly optimized and handle a huge amount of traffic, so logging was taking up something like 15% of all CPU time (and since they were CPU-bound, 15% of cost), so we have stats like those described in this article instead.

This is a false dichotomy. If you log things you don't need brilliant engineers spending countless hours on a logging infrastructure. All you need to do is log things to local disk, and delete some when you start to run out of space. You can write a fairly trivial script to do so.

That limits your logging rate to your available disk bandwidth.

That's not a problem for 99.99% of applications.

You could set up something to allow a dynamic change in logging level. Doesn't help with blip type issues that happened in the past, but helps with issues that arise and continue.

There's a defacto standard for many daemons using SIGUSR1 as a prompt to close/reopen log files. You could settle on some other non-used signals to change the log level.

I quite agree with this. In extreme circumstances, you don't know what you need until some disaster happens.

Consider why there are cockpit voice recorders and flight data recorders. Almost all of that data is thrown away.

In a more extreme example than the newspaper, consider some sort of data breach investigation.

My primary disagreement with that thesis would be the utility of logging things that clearly represent transactions, especially to the outside world.

eg, HTTP access logging is good and useful (vs non-structured debug timings on every fragment rendering)

eg, A mail server reporting delivery of a particular message into a mailbox.

I think your "events with identity" description is a nice succinct version of this same idea.