Large financial organizations have teams of analysts whose entire job is to relate consumer information, such as credit scores, to statistical outcomes. If the credit scores weren't giving them useful information, that would show up rather quickly.

I’m suddenly reminded of the ratings used to fuel the subprime mortgage crisis.

That was something subtly different. The banks more or less knew they were taking poor risks, the problem is it wasn't their risk to take. They quietly packaged these bad mortgages along with others and resold the mess to less sophisticated investors (like pension plans). Then the bubble burst, the well dried up, and banks started collapsing. It wasn't a mismanagement of risk, it was criminal fraud writ large.

I would argue that credit bureaus leaking all your information to criminals and then charging you money to protect you from those criminals is also criminal fraud writ large.

That doesn't seem relevant to the question of whether credit scores are statistically useful.

We've established that sufficiently advanced fraud can overcome the ability of organizations to vet the info that’s given to them. If the credit bureaus are actually organizations dedicated to fraud, then the fact that many other organization fall for it doesn’t tell us too much about their accuracy.

If Osama bin Laden's ghost appears to you and says "There will not be a terrorist attack in New York tomorrow.", and then there is, and this repeats a few times, the fact that his statement is, facially, a lie, and the fact that he can't even exist doesn't matter. You have an input that correlates with a certain output. The intent of the person who provided you the input, or even how nonsensical you think the input is, doesn't change its statistical usefulness.

So how did all these sophisticated financial organizations get taken by subprime mortgage bundles in 2008?

They didn't. Pension plans and individual investors are not sophisticated financial organizations. They rely on those that are to do their job in a non-fraudulent manner. They didn't.

Bear Stearns? Lehman Brothers? AIG? The hundreds of banks that went under?

Committed or were complicit in the fraud and got in too deep to recover from the inevitable crash. This has all been widely and loudly reported on for a decade. There's a reason people are pissed that the executives who ran these institutions into the ground and fucked up the economy were barely prosecuted.

I'm sorry, but is the reason that Equifax isn't in the same class as these guys because Equifax was totally upstanding, never cut corners or fudged numbers and were absolutely upfront when this security breach happened? Because if that's so then I'm curious about the breach they detected three month prior that they tried to cover up.

Equifax is just as shady as those lenders - more so IMO because they have absolutely no obligation or business relationship directly with the individual's whose private data they compromised.

So in addition to the info being valid or the organization evaluating it being too stupid to evaluate it correctly, we also have the possibility that these organizations are involved or complicit in the fraud.

Why am I supposed to take it as a given that if these organizations use the info it must be useful, then?

If it's not, they're wasting money for no reason, and someone will come along and do the same thing without giving the credit bureaus any money. They're not actually legally obliged to use credit bureau data. They could just throw money at people in the street and hope they get confused and throw back more.

A bank isn't an individual who is lending his own money. It is an organization that lends other people's money and needs a veneer of justifications for the choices they make, and credit reporting agencies are a perfect choice. You mentioned earlier that banks have statistical teams making these analyses, but most simply don't because that provides counter-information that undermines the whole value of credit reporting agencies (which is that they outsource the politically fraught choices and makes it easy to say "well we just followed their ratings, just like everyone else").

Regarding the subprime crisis, the biggest victims were the largest banks -- the most sophisticated being put completely out of business -- so not sure what the bit about pension plans comes from (many of those made a lot of money on it).