I don’t think regulator nepotism is the main reason that the authorities would be uncomfortable with loan decisions being made by a system that definitionally reinforces existing biases and is incapable of thought. It’s just a bad idea!

Well, by definition in these cases it's making a correct decision but that decision isn't popular. If red-lining wasn't profitable the banks would have gone broke and we wouldn't be talking about it.

The models aren't wrong, or even wrongly biased. We're systematically failing to ask the right questions because it's not politically correct to even call out the issues we're trying to combat. If you're trying to achieve "equity" then you need to define it and represent it in your equations, not ban various metrics until justice happens.

You don't need to check a random sample. You can have a policy where every single loan application is checked by a human, and you can add whatever affirmation is needed. It will still increase the productivity of those loan officers by a factor of 5. (Put it differently, banks would be able to lay off 80% of their loan officers).

The fun part is when a LLM hits that small probabiliy where they decide to go 'offscript'. It can result in a beautifully terrifying cascade of grammatically acceptable nonsense, and joe fun it would be in a legal document. We go from a loan for a home to a generous offer that includes a unicorn, a purple dishwasher, a unicorn, and a few dangling participles at the going market rate, all for the low low rate of 555.555.1212. [END TOKEN]--- WASHINGTON, D.C. President Trump today met with lawmakers to

I laughed really hard -- after trying to make sense of your comment!

Thank you for posting this :-)

I think the hope is that as LLMs get larger these issues will go away. Certainly there are issues with GPT-2 that completely went away when moving to larger models.

Honestly, I haven't even seen GPT-3.5-turbo exhibit this behavior myself, although I am willing to believe it could happen. Llama 7B, however, goes off-script constantly.

This is the exact kind of dystopic thinking that is feared with the use of AI.

"We regularly take randomized samples and have not found error, your appeal has been denied."

I mean come on, ethics anybody??

So long as it is better than the thing it replaces, I don't get the big deal.

Anonymizing risk is not a better system. Think a few consequences down the line...no one knows who theyre lending money to or from, its all figured out by AI/LLMs and as long as the incident rate is below a certain percent in a random sampling it doesn't matter what the errors are. WCGW?

Deleting discretion from the current system doesn't fix or improve it. Discretion is a really important part of the current system that prevents it from becoming more insane than it already is.

"better" is not a very clear criteria

It’s not though? Deterministic non black box models are superior unless the only thing you’re after is a coherent(ish) reply in English.

We can audit and fix the code of our existing laws as we slowly come to consensus, but how do you meaningly iterate on the output of an oracle in a way that converges to the societal consensus of societal structure?

This is true, but is it actually better?

You mean you could ask a LLM to look at a sample of the loans and decide if there was a bias.