I'm saying that your company's CTO, especially one with a very large companies, could likely identify two or three engineers who they pull into a meeting and say "reach out to these guys and get them whatever they need. Here's my cell, call me the moment you need the plane or additional resources."
Seriously, if a CTO has a budget of a few hundred million dollars and thousands of dedicated employees, how hard is it to throw a few crumbs to the open source community to change this situation from being one of a burden on a volunteer effort to, instead, one where they feel like they're in the middle of an international event where their knowledge and services are vital to keeping the internet alive?
Again, I'm exaggerating, but you see where I'm going with this. It's a missed opportunity for some seriously great PR out of a seriously bad situation.
It's not hard to extend this to the kind of software security flaw here. If I'm a developer on a package with a critical security vulnerability that needs to be fixed now, sending me extra developers who know absolutely nothing about the code I'm working on isn't going to be helpful--it's just going to waste my time trying to bring them up to speed (or more often, telling them to just go away). If I actually need help, I'll ask the people who I know can help me; trying to sift through unsolicited help to figure out who actually has the skills to do so would take too much time.
So think hard about what help Google et al could actually be providing to help log4j here. If you have to resort to clear exaggeration to find examples... maybe that's a sign that there actually isn't all that much that they could be doing that would actually be helpful.
I think you make a perfectly valid point and one that shouldn't be overlooked. How about this:
"Here's a $100K and an isolated penthouse suite down the road rented for the month where you can focus on fixing the problem and not be interrupted by screaming children. Here's a phone number if you need to delegate any specific tasks to additional teams."
Incentive to help. No added pressure. Just one practical example.
If children and family were viewed as too much of a distraction, I’m pretty sure the CTO (in this scenario) would simply choose a developer who lacks those distractions.
Let’s say the engineers chosen do have family. Why wouldn’t the company just comp a room in a local hotel?
In this instance either would be reasonable.
Have a team handle all github issues and media inquiries. Another team focus on initially evaluating all incoming pull requests to check for egregious errors or applicability.
Only after making it through the gauntlet would the original maintainers need to read and/or respond to them.
Especially when such overwhelming public attention and pressure overtakes a relatively small team like this one.
As with openssl, what needs to be done is that these volunteers need to be given cash so this is more than just a volunteer project. If a particular corporate entity doesn’t want to sponsor some of the maintainers to work on it full time, then the project needs full-time sponsorship by the Linux Foundation, ASF or under the OpenJDK.
But that is not to say that nothing is being done. At Google, at least, there are organized efforts staffed with plenty of people that are trying to solve the much much much bigger problem of "secure all of our open source dependencies and all future dependencies" rather than the individual problem of "secure this one dependency."
And PR? Google has been running projects like OSSFuzz for years and I haven't really seen it materialize as a large amount of positive PR, even in the tech community.
Google's Project Zero is both very helpful and gets them A LOT of PR, both tech and mainstream.
If all your IT projects have an RCE vulnerability that’s relatively easy to exploit, that should keep you up at night.
The actual solutions here are at a more abstract layer than individual vulns.
A week ago, this vulnerability might have been known at most to a few three-letter agencies. Today, every two-bit script kiddie will be trying to exploit it. It's not hard to see how the situation has changed.
Apache has processes for this, and project maintainers pointed people in that direction repeatedly (e.g. https://github.com/apache/logging-log4j2/pull/608#issuecomme..., https://github.com/apache/logging-log4j2/pull/608#issuecomme...).
The Apache foundation receives funding from a large number of organizations already: https://www.apache.org/foundation/thanks.html
Perhaps the right question to ask here is: what did Apache do to help their members in this event?
You can ask this question of the Apache foundation independently, without adding pressure on the project maintainers at this time.
"Let's get 3 of your best guys in touch with the guys over at log4j...."
As if you can just wire up some engineers to reach out to each other on unrelated projects and magic will happen.
Adding random new people generally does not immediately speeds up the process, but it is actively harmful in the middle of crisis.
Surely the difference is you are getting paid, and if your boss says, help these guys out, you can do it? As opposed to some guys with jobs who have a project on the side. The big guys could even do something like offer to pay the maintainers and maybe they can take leave or something.
I agree with both sentiments. The big guys are under no obligation to fix an issue in some library they happen to use. But the log4j guys are under even less obligation when they do it in their spare time.
Everyone should enjoy their weekends.
No, I'm not getting paid. What leads you to believe in that? My targets are defined yearly and are very well defined, and patching random FLOSS projects is not one of them. And what leads you to believe that others, such as my boss, don't have their own milestones to meet, and instead take random FLOSS requests from random people on the internet?
A FANG is not a magical entity where any engineer can drop everything they're doing at the drop of a hat to work on external projects, let alone one whose only possible outcome is at best total indifference and at worse we get the company to own a problem affecting everyone for no reason whatsoever.
Something I believe we agree on is that it is in the interest of large tech companies to spend time fixing critical security bugs in their own programs, regardless of who originally wrote the malfunctioning code and for whom said code was written.
One way to fix those bugs would be to create a patch for the external OSS library in instances where such a library is the origin of the vulnerability. This is especially practical when that library is used heavily as a basic piece of the company's common software development framework.
GP appears to be arguing that these patches should be upstreamed instead of simply being maintained internally until the bug is patched by someone else in the OSS community.
If all our projects suffer from a known RCE vulnerability, I’m fairly certain my boss will be happy for me to drop everything to get it resolved.
That’s not nearly at FAANG, but it’s certainly not an undue burden on the schedule.
If the big guys don’t want to do that, fair enough. But the open source maintainers are not under any obligation to work to anyones time lines either.
Your "(...)" elides the word "help," which completely changes the meaning of the quote, and your reply is constructed uncharitably as if that word wasn't in the original statement.
> Log4j's maintainers have a far more difficult and challenging job than FANGs
You are saying that the companies that built advanced ML-based Chess/Go engines like Alpha Zero/Go can't solve a simple logging bug involving string substitution?
If your company ends up using the product in all your teams/project and products wouldn't it be in the company's interest to keep the product safe?
How do we know you're not a CTO/C--/manager in your 'faang' just taking this opportunity to bitch about how bad and unreliable open source is? You do have a track record when it comes to this.
> I mean, are you willing to spend your weekend working around the clock to fix my problems?
Wow, that's cynical even for a 'faang' dude.
But as a professional representative of your org. surely you'll recognized the unsustainability of the situation and that it's far from ideal even in the pure self-interest of the company in question.
FAANG engineer here, and one who had to work extra hours to redeploy services with the log4j vulnerability fix. I'm not sure you understand the scope and constraints of this sort of problem. Log4j's maintainers have a far more difficult and challenging job than FANGs or any other consumer of a FLOSS package, who only need to consider their own personal internal constraints, and if push comes to shove can even force backwards-incompatible changes. The priority of any company, FANG or not, is to plug their own security holes ASAP. Until that's addressed the thought of diverting resources to fix someone else's security issues doesn't even register on the radar. I mean, are you willing to spend your weekend working around the clock to fix my problems? Why do you expect others like me to do that, then? Instead I'm spending a relaxing weekend with my family with the confort of knowing my service is safe. Why wouldn't I?