Banks and credit card issuers all pay hefty sums of money to the reporting agencies to use in underwriting their loans. They wouldn't do this if they weren't helpful in modeling risk.

I was speaking to a loan officer recently, and the purpose of the credit score seems only vaguely related to how well it actually predicts risk.

The gist was that if all their rates are based on a standard formula with no individual discretion, they are safe from regulation around (1) discrimination, (2) risky lending such as led to '08. That sounded more important to them than actually accurate risk modeling, especially since their competitors are all using the same formulas.

(edit: In particular, even if you offered them access to much more effective/predictive data points than the credit score, they would not use them because of (1) and (2) above.)

The majority of people on this board work in tech - just because money is exchanged for a service does not mean that the service purchased is worth that money... so can many attest about terrible highly paid consultants.

Large organizations are astoundingly good at finding inefficient ways to spend money.

Large financial organizations have teams of analysts whose entire job is to relate consumer information, such as credit scores, to statistical outcomes. If the credit scores weren't giving them useful information, that would show up rather quickly.

I’m suddenly reminded of the ratings used to fuel the subprime mortgage crisis.

That was something subtly different. The banks more or less knew they were taking poor risks, the problem is it wasn't their risk to take. They quietly packaged these bad mortgages along with others and resold the mess to less sophisticated investors (like pension plans). Then the bubble burst, the well dried up, and banks started collapsing. It wasn't a mismanagement of risk, it was criminal fraud writ large.

I would argue that credit bureaus leaking all your information to criminals and then charging you money to protect you from those criminals is also criminal fraud writ large.

That doesn't seem relevant to the question of whether credit scores are statistically useful.

We've established that sufficiently advanced fraud can overcome the ability of organizations to vet the info that’s given to them. If the credit bureaus are actually organizations dedicated to fraud, then the fact that many other organization fall for it doesn’t tell us too much about their accuracy.

If Osama bin Laden's ghost appears to you and says "There will not be a terrorist attack in New York tomorrow.", and then there is, and this repeats a few times, the fact that his statement is, facially, a lie, and the fact that he can't even exist doesn't matter. You have an input that correlates with a certain output. The intent of the person who provided you the input, or even how nonsensical you think the input is, doesn't change its statistical usefulness.

So how did all these sophisticated financial organizations get taken by subprime mortgage bundles in 2008?

They didn't. Pension plans and individual investors are not sophisticated financial organizations. They rely on those that are to do their job in a non-fraudulent manner. They didn't.

Bear Stearns? Lehman Brothers? AIG? The hundreds of banks that went under?

Committed or were complicit in the fraud and got in too deep to recover from the inevitable crash. This has all been widely and loudly reported on for a decade. There's a reason people are pissed that the executives who ran these institutions into the ground and fucked up the economy were barely prosecuted.

I'm sorry, but is the reason that Equifax isn't in the same class as these guys because Equifax was totally upstanding, never cut corners or fudged numbers and were absolutely upfront when this security breach happened? Because if that's so then I'm curious about the breach they detected three month prior that they tried to cover up.

Equifax is just as shady as those lenders - more so IMO because they have absolutely no obligation or business relationship directly with the individual's whose private data they compromised.

So in addition to the info being valid or the organization evaluating it being too stupid to evaluate it correctly, we also have the possibility that these organizations are involved or complicit in the fraud.

Why am I supposed to take it as a given that if these organizations use the info it must be useful, then?

If it's not, they're wasting money for no reason, and someone will come along and do the same thing without giving the credit bureaus any money. They're not actually legally obliged to use credit bureau data. They could just throw money at people in the street and hope they get confused and throw back more.

A bank isn't an individual who is lending his own money. It is an organization that lends other people's money and needs a veneer of justifications for the choices they make, and credit reporting agencies are a perfect choice. You mentioned earlier that banks have statistical teams making these analyses, but most simply don't because that provides counter-information that undermines the whole value of credit reporting agencies (which is that they outsource the politically fraught choices and makes it easy to say "well we just followed their ratings, just like everyone else").

Regarding the subprime crisis, the biggest victims were the largest banks -- the most sophisticated being put completely out of business -- so not sure what the bit about pension plans comes from (many of those made a lot of money on it).

"They wouldn't do this if they weren't helpful in modeling risk."

This isn't necessarily true.

A counterargument is that credit scores offer banks an easy way to outsource what often ends up being a very contentious, politically-fraught process. Yet many analyses have found credit scores barely better than random dice -- the guy with the perfect credit score has a perfect credit score, until he doesn't and there is a wake of delinquency in his wake.

It's also helpful to assess real world motivations. Extraordinarily few of the employees, including at the executive level, at a bank are legitimately concerned about the long term risk to the bank. Success is measured at the quarterly interval, and if you can justify your actions on a quantifiable measure -- even if it's a measure that has little predictive value -- then that's just perfect.

The biggest indicator that someone is a credit risk is that they are maintaining or growing higher interest borrowing products, such as carrying a balance on a credit card. This is an absolute flashing light indication that someone is over-extended, yet the credit monitoring agencies would not bite the hand that feeds them by making too big of a deal about this. Indeed, gross over-borrowing is barely a blip on a credit report, because the people who lend the money ensure that it isn't. The world is absolutely awash in cheap cash and banks are desperate to lend it.

In the wake of the subprime crisis everyone said "oh yes, of course there's a problem there it was the credit agency that was just marking these all wrong", but exactly the same thing is happening on personal credit reports. Of course it is, because the credit reporting agency is there to legitimize whatever the bank wants to do.

This is a fallacious argument. Just because banks use credit scores doesn't mean they are accurate (see argumentum ad populum).

What we'd need to demonstrate your claim is for example some data linking credit scores at time of loan to default rate, adjusted for income and loan terms.

Maybe just go upthread to patrick's comment on the topic. He's a much better writer than me.

https://news.ycombinator.com/item?id=20580478