The larger issue is in javas dependency system are built with dependencies on these libraries, so if there is no commons logging (or bridge, ala slf4j), then the program won't run.
There are a few other techinical traps with the jvm, like using system.out logging can cause issues with threaded code, or having one giant static logger being imported all over the place conveniently (this is before inversion-of-control became the dominant paradigm) had a few shortcomings, especially as programs grew. Ultimately, though, if you were going to use some third party libraries or app servers, you were almost certainly going to get pulled into the commons-logging vortex, which means you're now configuring that AND your homespun logger.
There is also the case for things like SOLR, where it might be just being used out of the box, but the distribution includes the affected JARs, even if you're just posting and retrieving documents with a python script doing print-logging.
Honestly, I didn't even know log4j2 was a thing, I've moved as much over to slf4j/logback as I could ages ago because of the madness of the JDK logging ecosystem.
Note: In case someone is tempted to say that it is trivial to plug slf4j over logback, it is for greenfield work. Replacing an existing logging library, or the use of multiple logging libraries in favor of slf4j over one lib is a confusing matter, mostly because the error thrown at runtime are misleading.
When Java finally added a logging library, they added one that no-one had ever heard of before, and which had fundamental flaws. That's why no-one uses it. At the time, log4j existed and was in wide use, but was not represented on the committee.
The author of log4j eventually decided to go a different way and created slf4j / logback. This offering is compatible with log4j and considerably simpler in many ways.
Having not been in the Java world for many years, I was surprised log4j was still in widespread use. But maybe I shouldn't be surprised - whenever we have to interact with Java teams, we always have to tamp down on the tendency towards complexity.
It's Java; the ecosystem is, politely, rather conservative and slow moving.
Logging facades like slf4j help a lot, but then you still need to implement the required interfaces. For something like the specific Log4J CVE, things would be significantly worse if it weren't centralized in a library. They have a bug in deserialization when talking to LDAP servers, allowing a compromised LDAP server to execute code remotely. It's likely that a subset of libraries would likely have rolled this functionality independently, and then you have many libraries that all need to be patched and updated instead of one.
Rolling your own logging library to reimplement it is not trivial, even if you ignore the more esoteric output formats.
Sure, in retrospect they went too far with some of the dynamic logging features in log4j2, but it is no exaggeration to say that log4j revolutionized logging in the Java world when it first came out, and heavily shaped later solutions like logback and java.util.logging.
Not to mention a wide range of destinations, too. Want to log to a database table? You’re covered. What about log messages to a chat server (xmpp, slack) or email but only fatal errors? You’re covered.
Now roll your own logging system that supports those destinations safely and get back to me to do pen testing. I bet we’ll find some vulns in your code.
Most of us have log4j in our system because it's used by some various dependency of one type or another, and up until now it has generally just worked.
> It’s dynamically configurable
It's a logging library. You don't reconfigure logging, you just make another logger and use it. And by the way, you take the configurations from your global configuration system, "dynamic configurable" is a feature of the configuration system, not of the logging.
> supports a wide range of output formats and destinations
Yeah, like network aware templates. (But the destinations are a feature, multiple destinations are one of the few things a logging library should support.)
Despite what people claim, the Java culture of loving complexity never did go away.
Prior company had a shared plugin that let you turn up the logging for a given logger remotely for 30 minutes (it self reset). Extremely helpful for incident debugging.
Now we can have an interesting discussion about whether that capability is worth the complexity, but that’s a very different discussion than unilaterally declaring that other people’s use cases just don’t exist.
> Despite what people claim, the Java culture of loving complexity never did go away.
Do you develop in Java? Because I do, and your takes don’t match my daily experience one bit. My experience going from Spring 3 to Spring Boot 2 is in fact one of reducing complexity (goodbye XML bean configuration!) and to more “it just works” situations than before.
And you really believe this should be a feature of you logging library, and not of you configuration system?
The only way to make a "reconfigurable" logging system is by having a stable facade that restarts everything behind it. But that extra layer does not belong on a logging system, there are plenty of other things you need to make "reconfigurable", and now you are adding an extra layer into each of them, instead of having only one on your main entry point.
Pushing that feature down into the logging system is the choice that maximizes complexity and minimizes functionality.
Also, I noticed you didn’t answer my question about whether you’re a Java developer.
I was once, but haven't used it professionally for a while. It's on my "I'd rather not" list, but not strongly so.
Oh, and now that you described it with so many details, the attachment to complexity does look more like addiction than love.
This is why I ask. Java has changed a lot since ~1.8, and it is super common to see ex or non java developers making grandiose and sweeping statements about a community and language that they are no longer part of. Having developed in Java on and off since ~1.7 and Spring 3, I can certainly say that I have seen a strong movement towards less complexity and more convention. It's not perfect by any stretch, but I think saying that the space is "in love with complexity" is incorrect.
You of course don't have to go and re-use Java on my suggestion. We all have our preferences, and that's ok! But I would generally recommend against such sweeping and dismissive general statements about a culture that you are no longer part of. Things change, and your experience back then is probably not representative of things today.
> Oh, and now that you described it with so many details, the attachment to complexity does look more like addiction than love.
There's a fine line between disagreement and insult, and you are over it.
> How much log data do you think cloudwatch and the other majors filter per hour?
How much do you think the major providers pay for that infrastructure per hour, and are you willing to shoulder that cost too?
(Hint: log ingestion is $0.50 per gigabyte in cloudwatch. So a terabyte of logs an hour is $512/hr)
Only if the emissions of said logs is doing more than stdout or fs writing and it’s not done in another thread.
> Log emission can often have a negative impact on performance, especially since peak logs and peak traffic tend to coincide. I’ve seen P95 latencies suffer just because of logs.
Yes and you need to fix that terabyte per hour garbage as that is well beyond typical for a single service. We’re also talking about java so 7000 page stack traces is where i’d start…
In terms of features, one that comes to mind log rotation.
I dunno, I suppose all together I’d answer your third paragraph to the affirmative — I’m more productive and the libraries improve significantly within crucibles like this.
That said, I do agree that there is a problem with large dependency trees. It seems building a simple hello-world app with Maven ends up downloading about 1/2 the internet. :)
There is also lots of stuff you might want a logging library to do that while not very hard would be annoying to do yourself for every project like log file rotation, switching log level while the application is running, optimizations (like in c/c++/rust you might compile for INFO level and drop all the DEBUG and TRACE level logging from the binary), different log levels for different parts of the app (including dependencies), etc
The problem with rolling your own is that, for a non trivial technology like a producionised logging framework or some encryption library, you're highly likely to write a massive security hole yourself. The great thing about battle-tested open source software, used by 1000s, is that years of real world use, development, bug fixes and general scrutiny from a large number of people makes for a high level of reliability normally.
Obviously here we have a problem with this approach but avoiding third party dependencies carries its own risks and obvious costs too. Let's focus on how reliable so many of these libraries seem to be and how rare problems of this scale are.
I think a bigger problem here is the fact that so many people are using so few libraries. Maybe we need more open source libraries to counter single points of failure. As with how we deploy our code on the same clouds and so on, single points of failure abound these days. Things fail however well they're engineered and therefore we need diversification of everything.
(1) Coordinating with dependencies requires at a minimum a common API (which, in a statically typed language, means a library providing at least the interface).
(2) Configurability to log to different destinations and with different formats, which you eventually tend to want on major or long-lived projects, can be fiddly, as can the specifics of different formats and targets. Each of the bits is simple, but added up it's a lot over time, and reinventing the wheel, the axle, the tire, and the air pump for it just doesn't make a lot of sense. Especially given the need for a library for #1, it's best to get the core functionality together with it, and the peripheral pieces either together or as plugins, and only solving unique problems, if you have them.
Log4J has been around 20 years. Log4J inspired and was lifted into the JDK. JDK logging is essentially an inspired copy of Log4J. The Sun coders were no better or worse than the Apache coders. I use JDK logging personally simply because it's one less dependency. JDK logging hasn't change since JDK 1.4, and is weaker than, well, pretty much anything else. But, with a simple wrapper I've used for 15 years, it does most everything I want from a logger.
That said.
Log4J has been, and is still, a boon to the Java community. Arguably, Log4J is the root of a tree of vast array of logging frameworks, across languages. Java server developers essentially live and die by their logs. It's routine for developers in dark rooms with screen lit faces to pouring through logs with endless stack traces. Thank heavens for Java stack traces.
Logging is part and parcel to the Java server side experience, and even the client side, and we can place much of that on the shoulders of giants like Log4J because it made logging easy and set the stage. It's so helpful, so useful, so flexible (obviously, perhaps, a bit too flexible), and so powerful.
Because Log4J inspired the other loggers like the JDK Logger, we have logging shims. Shims like Commons Logging, that act as intermediaries that can have adapters written so that we can use other logging frameworks. Log4J was one of the first, was, and is still, dominant in the community, but it's not alone, and thus a bad choice for things like libraries. Instead, those choose the shims like Commons Logging that developers can use to configure to route through Log4J or JDK logging or any of the others. That said, even programs that use Log4J directly can be routed to other loggers.
This is all entrenched. It's part of the flavor of server side Java, configuring the different logging shims to write to your logger of choice on your system. Just the way it is. But, that's what happens when you don't live in a mono-culture. Feature, not a bug, and it helps empower the vast array of software that millions of developers and applications rely on everyday. Nobody designed it this way. It didn't start this way, it just evolved this way. It's a very "Java" thing.
I watch Stack Overflow questions and 90+% of the time when someone asks "How do I" what they mean is "What library do I need", not "How can I write this". This is the sign of the times, and Java is not alone. The beauty of Java is that it made this kind of sharing REALLY easy. REALLY REALLY easy.
Apache has a solid reputation for good projects and good code and good stewardship. It's not a back alley transaction to grab an Apache Java jar file and shove it in your project. Is it all perfect? No, but what is?
Transitive dependencies for one thing. I rather enjoy having my DB driver, connection pool, kafka client, and web framework all emit their own logs, which is not possible if I wrote my own logging library.
> I get that the tradeoff it is worth it for complicated things (e.g. crypto libraries). But logging, really?
Logging at scale is more complicated than people give it credit for. Personally I've had my services suffer noticeable performance degredations due to poor logging framework config. We've ended up with a fairly complicated logging config that allows us to carefully balance collecting as much developer useful information as possible against the need to drop excessive logs during periods of heavy activity. Oh, and it's gotta be converted into JSON for aggregation into our observability tools, and decorated appropriately so that our logs can be correlated with our application traces, because that is incredibly useful during incidents.
Oh, and it all needs to work no matter what threading configuration you throw at it. That matters a lot too.
Do we need all those things? Arguably no, but then again they are helpful tools to have. Without them, the team would have a harder time developing on our systems. You have to understand the use case that people need to fulfill before dismissing the entire thing as unnecessary and pointless.
> Software development culture today is too quick to adopt a huge tree of dependencies of unknown quality, rather than thinking about how to minimize dependencies to only those truly necessary. The leftpad fiasco was but an extreme example of this, but I see it all the time, and it seems probable that there are hundreds (maybe even thousands) of similarly severe problems out there in widely used dependencies that we just don't know about yet.
I see this argument a lot, and I think it's a great example of "grass is greener" syndrome. People don't think through the practical consequences of drastically reducing their library usage. I personally worked on a project that had basically no external libraries due to the language choice, and we wasted inordinate amounts of time trying to chase down bugs we wrote in things like our database library, localization library, and yes, our logging library. Looking back I can say with very strong confidence that what we were doing was a massive waste of the company's resources, and rewriting it into Java or C# would have been the right call, even with the complexities those ecosystems bring to the table.
Yes, using external libraries comes with tradeoffs. Fixing bugs like this is one unpleasant possibility. But let's be clear that the alternative is a massive drop in developer productivity to write basic functionality that already is implemented elsewhere. Having been on both sides of this equation, I can confidently say that you'll spend way less time fixing security issues as they arise than you will reinventing the metaphorical wheel.
(I do agree that leftpad is an example of going way too far with libraries, but I think the library culture for NPM and the Java ecosystem are quite different).
A lot of these libs separate logging into layouts and parameters. If you log input parameters as layouts. You might break the logging. I have seen this at every company I worked with.
The only really surprising thing about log4j is the remote code execution.
yea literally any library/dependency can introduce risk, including stuff coded internally for those purposes
Sir, you might have just extended all of our careers by at least a decade. We salute you.
I agree that developers are too quick to put themselves at the mercy of 3rd parties, but it's a risk-dollar trade off we'll continue to make because reinventing the wheel each time would sap our margins.
The latter option represents a liability, which in this case (as with others) has shown can be a tremendous risk. Is the time saving really worth it, at the cost of risking disasters like this?
I get that the tradeoff it is worth it for complicated things (e.g. crypto libraries). But logging, really?
Software development culture today is too quick to adopt a huge tree of dependencies of unknown quality, rather than thinking about how to minimize dependencies to only those truly necessary. The leftpad fiasco was but an extreme example of this, but I see it all the time, and it seems probable that there are hundreds (maybe even thousands) of similarly severe problems out there in widely used dependencies that we just don't know about yet.