So long as it is better than the thing it replaces, I don't get the big deal.

Anonymizing risk is not a better system. Think a few consequences down the line...no one knows who theyre lending money to or from, its all figured out by AI/LLMs and as long as the incident rate is below a certain percent in a random sampling it doesn't matter what the errors are. WCGW?

Deleting discretion from the current system doesn't fix or improve it. Discretion is a really important part of the current system that prevents it from becoming more insane than it already is.

"better" is not a very clear criteria

It’s not though? Deterministic non black box models are superior unless the only thing you’re after is a coherent(ish) reply in English.

We can audit and fix the code of our existing laws as we slowly come to consensus, but how do you meaningly iterate on the output of an oracle in a way that converges to the societal consensus of societal structure?

This is true, but is it actually better?