Re:LLMs Are Bad Judges. So Use Our Classifier Instead
by adlumal on Monday September 29, 2025 @12:08AM
(#45410250)
Having done some work in the legal AI field, I wonder how this classifier deals with issues of transparency, explainability and ultimately trust? It’s valuable to have some idea of how a proceedings might unfold but from my experience most competent lawyers have a high bar when it comes to trusting any AI/ML output.
Re:LLMs Are Bad Judges. So Use Our Classifier Instead
by Taikonerd on Monday September 29, 2025 @08:26AM
(#45412836)
I was worried about explainability, too. If the classifier just spat out "INNOCENT" or "GUILTY," it would be useless -- the legal reasoning has to be part of the output.
Looking at the paper, the classifier definitely does output its reasoning:
"The legal issue at hand is whether the 50/50 royalty split in the 1961 contract binds only pre-existing affiliates or if it also includes affiliates that come into being after the agreement..."
Re:LLMs Are Bad Judges. So Use Our Classifier Instead
by lordgrenville on Monday September 29, 2025 @07:22AM
(#45412388)
True, but they own that:
> Declaration of Interest: [Authors] have financial interests in...Arbitrus.ai. As the title would suggest, the authors are making no effort to obfuscate this fact.
Re:LLMs Are Bad Judges. So Use Our Classifier Instead
by Taikonerd on Monday September 29, 2025 @08:30AM
(#45412890)
I had thought: what's the business model for Arbitrus? Is it going to be a sort of "suggested finding" tool for judges? Or are law firms going to use it to screen cases, so they can pick winners?
It seems like the answer is neither: on their website, Arbitrus.ai says it's for private arbitration. "Arbitrus is a private court system with an AI judge. Why use the public court system or expensive AAA arbitration to settle your disputes, when you can do it faster, cheaper, and better with Arbitrus?"
Re:LLMs Are Bad Judges. So Use Our Classifier Instead
by barbazoo on Monday September 29, 2025 @12:31PM
(#45415719)
The margin and line spacing makes this hard to read. Is this how you're supposed to typeset a paper? Some pages have three, maybe four sentences on them.
Looking at the paper, the classifier definitely does output its reasoning:
"The legal issue at hand is whether the 50/50 royalty split in the 1961 contract binds only pre-existing affiliates or if it also includes affiliates that come into being after the agreement..."
> We built one called Arbitrus. We put it through a mini-Choi test and it mopped the floor with the competition
> Declaration of Interest: [Authors] have financial interests in...Arbitrus.ai. As the title would suggest, the authors are making no effort to obfuscate this fact.
It seems like the answer is neither: on their website, Arbitrus.ai says it's for private arbitration. "Arbitrus is a private court system with an AI judge. Why use the public court system or expensive AAA arbitration to settle your disputes, when you can do it faster, cheaper, and better with Arbitrus?"
Even LLMs can be viewed as classifiers, as the paper (ad?) itself admits.