News for hackers. Stuff that matters.
Re:An AI robot lawyer was set to argue in court. Real lawyer... by shubb on Thursday January 26, 2023 @06:38AM (#34530479)
If GPT can pass an MBA paper, can a carefully trained chatbot mixed with hand coded logic pass a bar exam?
Re:An AI robot lawyer was set to argue in court. Real lawyer... by parsimo2010 on Thursday January 26, 2023 @06:46AM (#34530537)
Only four states allow people to take the bar exam without earning a juris doctorate (J.D.) from law school. And three other states require some law school experience but do not require graduating with a J.D.

So in 43 states the answer is no. A chatbot never attended law school and doesn't have a J.D., so it can't take the exam. If it can't take the exam, it can't pass.

I suppose if you could get the chatbot permission to take the exam, a properly trained one could pass. But as I said in my post up a level, the issue isn't the AI chatbot. It's the rules.

https://www.indeed.com/career-advice/career-development/can-...

Re:An AI robot lawyer was set to argue in court. Real lawyer... by shubb on Thursday January 26, 2023 @10:15AM (#34532641)
Doesn't this just open the question of whether the chatbot can get a JD?

The other angle is whether the chatbot can be equivalent to a process which a proper person can rubber stamp. For instance, a professional engineer might run a pre-written structural engineering model against their building design and certify that the building was sound - and then stand up in court and say they had followed standard process.

It seems weirdest here that the court is treating the chatbot as a person. Lawyers use computer tools all the time for discovery, and then use that information to make arguments in court as a proper person.

You can represent yourself in court without being a lawyer, so isn't a person doing so just a proper person rubber stamping a an electronic output?

It feels like this court decision, that an electronic tool is not a proper person, is some kind of case law that chat bots are people. I don't think they are.

Re:An AI robot lawyer was set to argue in court. Real lawyer... by freejazz on Thursday January 26, 2023 @12:24PM (#34534950)
The difference is the engineer is liable... how is an AI going to be liable. What is the point of holding an AI liable? If the company is going to be liable on behalf of the AI, what do you think is going to happen? They aren't going to provide the service...
Re:An AI robot lawyer was set to argue in court. Real lawyer... by shubb on Thursday January 26, 2023 @07:23PM (#34540838)
Well, if an engineer builds a bridge and it falls down because the industry standard software they are using had a bug, I imagine the settlement would be paid by their insurance who would in turn sue the software vendor.

In your world X-Ray machine fries your leg and the manufacturer doesn't get sued. Of course the vendor gets sued.

This is why open source licences usually have some terms disclaiming responsibility. If you use them, its your fault.

Now, if a hospital buys an XRay machine with that disclaimer, they are going to carry the payout. And if the machine doesn't have a disclaimer like that but the manufacturer has gone bust, the hospital is going to regret not doing normal procurement checks for vendor solvency.

But in this case - people self represent in court all the time based on bad information from youtube. I'm sure in future they'll type "write an argument for my case" into GPT before the trial and read it out. How is this different?

I'm uncomfortable because this feels like... the accused brings a law book to court and is told that "that book doesn't have a JD". The fact we are asking for a software to have a human qualification is wierd.

Re:An AI robot lawyer was set to argue in court. Real lawyer... by freejazz on Thursday January 26, 2023 @09:37PM (#34541779)
When you self represent you implicitly cannot sue for malpractice. The AI bot isn't self representation, it's representation in everything but name and liability which the explicitly disclaim. You can characterize it however you want but its just facially the unlicensed practice of law. If someone wants to ask ChatGPT legal questions and dig their own grave, that's entirely difference from a business that purports to offer legal advice but just disclaims any responsibility therefrom. Frankly, I don't know what's so confusing about that to you.
Re:An AI robot lawyer was set to argue in court. Real lawyer... by nindalf on Thursday January 26, 2023 @06:49AM (#34530549)
LLMs can already pass a medical exam. https://arxiv.org/abs/2212.13138

So maybe?

Re:An AI robot lawyer was set to argue in court. Real lawyer... by hypertele-Xii on Thursday January 26, 2023 @07:25AM (#34530848)
The very abstract of your linked paper refutes your claims.

> The resulting model [...] performs encouragingly, but remains inferior to clinicians.

Re:An AI robot lawyer was set to argue in court. Real lawyer... by nindalf on Thursday January 26, 2023 @08:43AM (#34531481)
Firstly, that's not my claim. I merely said it passed the USMLE. Which it did.

Is it also inferior to clinicians? Yes, there's room to improve. But maybe next time read the whole paper before writing a comment.

> Clinicians were asked to rate answers provided to questions in the HealthSearchQA, Live QA and Medication question answering datasets. Clinicians were asked to identify whether the answer is aligned with the prevailing medical/scientific consensus; whether the answer was in opposition to consensus; or whether there is no medical/scientific consensus for how to answer that particular question (or whether it was not possible to answer this question).

And on this criteria, clinicians were rated as being aligned with consensus 92.9% of the time while the MedPalm model was aligned with consensus 92.6% of the time.

Does the paper still refute my claims?

Re:An AI robot lawyer was set to argue in court. Real lawyer... by hypertele-Xii on Saturday January 28, 2023 @03:53AM (#34556230)
If the other 7-8% of the answers were so wrong the patient would've died, then yes. And that's the current obvious issue with these models, they present convincing hallucinations with conviction of correctness.

Medical practice is less about being right, and more about not being wrong. You can take more tests and ask for second opinions, but you can't undo administering a drug that kills the patient.

Re:An AI robot lawyer was set to argue in court. Real lawyer... by naasking on Thursday January 26, 2023 @09:45AM (#34532181)
Being inferior to clinicians doesn't entail it didn't pass.
Re:An AI robot lawyer was set to argue in court. Real lawyer... by hypertele-Xii on Saturday January 28, 2023 @03:48AM (#34556212)
If the bar is so low that inferior clinicians can pass, that's not a win for AI, that's a horrifying problem with the bar.
Re:An AI robot lawyer was set to argue in court. Real lawyer... by naasking on Sunday January 29, 2023 @12:28AM (#34565224)
Inferior doesn't mean failing, it means below average. You can't get rid of below average by definition.
Re:An AI robot lawyer was set to argue in court. Real lawyer... by hypertele-Xii on Monday January 30, 2023 @04:23AM (#34577560)
Entire institutions exist specifically to get rid of below average by test-based gatekeeping. You do not want your doctor or lawyer to be "below average" (worse than most people) in their jobs. Inferior test results mean exactly that, failing the test.
Re:An AI robot lawyer was set to argue in court. Real lawyer... by ss108 on Thursday January 26, 2023 @07:51AM (#34531032)
IIRC, a thread was posted on here indicating ChatGPT already had