This sounds to me like they were writing it into some log analysis tool, and people happened to pull it up.
I have no knowledge of FB internals, but as an example let's say they have Splunk, which is a popular distributed log analysis tool (or some custom equivalent, because FB is huge). Splunk makes it very easy to pull up tons of logging for any app - that's what it's for. So write a poorly-constrained query and you can get millions of log lines from many different apps.
It's also feasible that everyone (or nearly so) in the company has access to Splunk.
So if somebody, somewhere accidentally writes passwords into a logfile, and it gets indexed into Splunk, a ton of people might theoretically see it, but just skim past it because it's not what they were looking for. Picture trying to hunt down an error, and wading through many screens full of logging from different apps - if there's a password in there you could easily not notice if you're not looking for it.
That's all speculation, but it's also entirely plausible. All it takes is accidentally logging out those passwords to somewhere that Splunk can see.
Also not trying to say this is in any way OK or acceptable, just that I can understand how a seemingly-small mistake could quickly result in thousands of people having access to passwords.
Have you ever worked at a software co, discovered <terrible software practice>, raised a flag, and were told "thanks, we've added a ticket to the backlog"?
I'd bet that some of them did flag it.
It's probably a bit more complicated than that. Usually the things that I encounter have to do with how HTTP requests are logged.
For example, putting sensitive information in a URL that's loaded over HTTPS is considered insecure because many companies have policies where they log every URL that their employees visit. (Think of a password reset link.)
A lot of inexperienced programmers don't realize this, because they don't realize that you can man-in-the-middle yourself, and that most corporate computers come preconfigured to allow the employer to man-in-the-middle everyone.
So, if a password reset link never expires, it means that some guy in IT can own an account that was reset on a corporate computer.
(This, basically, is how they catch people viewing porn on their work computers.)
Anyway, my point is that the problem is probably something where a junior programmer transmitted a password in a way that they didn't realize was being logged.
If that is your experience, then that's a truly wonderful thing.
Might it be possible that at many companies, teams with deadlines to hit will tend to prioritize feature work over details like this? Perhaps especially so when teams are not rewarded for fixing vulnerabilities and are punished for not meeting deadlines? Particularly when the actual bug at hand is that the full contents of a POST are being logged, and a PM might not read the ticket enough to understand that this includes a password...
Again, you're completely right in every way about what should happen. It just could be possible that this could reflect something other than all experiences all software engineers have had.
For you maybe. For Facebook obviously it's different.
What evidence do you have for that? Nowhere in the article does it say facebook "explicity logged" passwords. The logging likely happened through some unintended and roundabout process that is far from explicit.
I think incompetence over malice is almost certainly the right answer here.
Nonetheless, I stick to incompetence over malice. There's just so much more of the former in the world than the latter!
GDPR may levy fines but ultimately it hasn't done anything to stop managers focusing on new features and sales income.
I would not be surprised. Some of those 9,000,000+ queries and 2,000+ employees must have had some nefarious use. Statistically speaking...
The issue is presumably that they had no exclusion to the logging for sensitive information like passwords, which is honestly very easy to overlook.
So two thousand Facebook employees were not "retrieving passwords". They were looking at the API logs, which is a normal thing to do.
Wait, no ? How is that fine and normal ? It doesn't seem fine and normal all.
Check the network tab in dev-tools when logging in to hn for example and you'll see your password in the request body :)
This imo is the truly alarming takeaway. FB employees were retrieving user passwords? Around two thousand FB employees? How in God's name is Zuckerberg going to perform his usual performative contrition about that one?
I'm just trying to imagine the data structures that were being retrieved from databases. Either they stored something like a big user account data type that contained their password in plaintext, which imo is a really weird design choice, or logs for other services were being mixed in with logs leaking the user/pass combos.
Surely one of the engineers could have noticed and said 'wait a minute... those are logins' over the course of the years? We hear all the time that people want FB to follow a responsible social practices (the debate on what those are rages on, which is great imo), but can't FB at least wrangle its own code base?
On the other hand, we shouldn't take the stance that heads should roll, imo - it would just create a chilling effect that would deter other companies from ever going public about their own security mishaps.
edit: I should probably tone it down in this comment but I'll leave it for posterity