It doesn't really matter if it'd work well or poorly. Lawyers don't want to be replaced, and being a lawyer entails a great ability to be annoying to delay/prevent things you don't want to happen.
One of those is that the lawyer arguing a case is properly credentialed and has been admitted to the bar, and is a professional subject to malpractice standards, who can be held responsible for their performance. An AI spitting out statistically likely responses can't be considered an actual party to the proceedings in that sense.
If a lawyer cites a non-existent precedent, they can make their apologies to the court or be sanctioned. If the AI cites a non-existent precedent, there's literally no way to incorporate that error back into the AI because there's no factual underlying model against which to check the AI's output--unless you had an actual lawyer checking it, in which case, what's the point of the AI?
Someone standing in court, repeating what they hear through an earpiece, is literally committing a fraud on the court by presenting themselves as a credentialled attorney. The stunt of "haha, it was really just chatGPT!" would have had severe legal consequences for everyone involved. The harsh response saved DoNotPay from itself.
IANAL, but I would bet the level of effort to fact check an AI's output would be orders of magnitude lower than researching and building all your own facts.
I used it to generate some ffmpeg commands. I had to verify all the flags myself, but it was like 5 minutes of work compared to probably hours it would have taken me to figure them all out on my own.
If it was AI-assisted lawyer, it would be a whole different discussion. Aside from requiring a live feed of interactions to a remote system and other technical details, “lawyers using supportive tools while exercising their own judgement on behalf of their client” isn’t controversial the way marketing an automated system as, or as a substitute for, legal counsel and representation is.
Fact-checking nonsensical output would take a lot longer than researching a single body of law, which you can generally do by just looking up a recent case on the matter. You don't need to check every cite; that will have been done for you by the lawyers and judges involved in that case.
But checking every cite in an AI's output: many of those citations won't exist, and for the ones that do, you'll need to closely read all of them to confirm that they say what the AI claims they say, or are even within the ballpark of what the AI claims they say.
Of course there could be a difference between an reading all the cases an AI says are important and actually finding the important cases including ones the AI didn't point you at. However this is not what the bet was about.
Actually fact-checking an AI requires finding and reading all the precedent yourself to veryify that the AI has both cited accurately and not missed contradictory precedent that is more relevant (whether newer, from a higher court, or more specifically on-point.)
If it has got an established track record, just as with a human assistant, you can make an informed decision about what corners you can afford to cut on that, but then you aren't really fact-checking it.
OTOH, an AI properly trained on one of the existing human-curated and annotated databases linking case law to issues and tracking which cases apply, overrule, or modify holdings from others might be extremely impressive—but those are likely to be expensive products tied to existing offerings from Westlaw, LexisNexis, etc.
To the cost. If it removed the need for one legal assistant or associate then anything less than the cost of employing said person would be profit. So if it cost < 50k a year you'd be saving. (cost of employing is more than just salary)
Why would this be different with an AI assistant to help you? It's not a binary "do or do not". Just because you have an assistant doesn't mean you don't do anything. Kind of like driver assist can handle some of the load vs full self-driving.
> But checking every cite in an AI's output: many of those citations won't exist, and for the ones that do, you'll need to closely read all of them to confirm that they say what the AI claims they say, or are even within the ballpark of what the AI claims they say.
But you'd have to do this anyway if you did all the research yourself. At least the AI assistant can help give you some good leads so you don't have to start from scratch. A lazy lawyer could skip some verifying, but a good lawyer would still benefit from an AI assistant as was my original bet, just like they would benefit from interns or paralegals, etc. And all those interns and paralegals could still be there, helping verify facts.
No, that's exactly the opposite of what I'm saying. If you did the research yourself, you wouldn't need to verify every cite once you find a relevant source/cite, because previous lawyers would have already validated the citations contained within that source. (A good lawyer should validate at least some of those cites, but frequently that's not necessary unless you're dealing with big stakes.)
And the AI assistant, at least this one and the ones based on ChatGPT, don't provide good leads. They provide crap leads that not only don't exist, but increase the amount of work. And any "AI" based on LLM will never be capable of providing good cites, because they'll never understand what they're reading and/or citing, and they'll miss relevant citations that are not statistically likely (i.e., new case law, or cases with similar facts, or similar law, or otherwise similar contexts that can be applied to the case at hand) that a context-aware AI or living breathing human would find easily.
At best, LLM-based AI might be able to help people with very simple legal situations. But you don't need AI for that. A single decision tree is easier to implement, and it's even easier to verify the domain-specific process and outcomes to make sure you don't get something silly like happened with this "AI".
I think making the connection between the predictive output and an underlying representation of reality is the next great step, but until that happens, chatGPT's output is just amazing mimicry of human language.
I have some past experience working in the courts in my state, and I know there are many judges who are perfectly fine with dismissing minor traffic infractions for no reason other than that they feel like it. If you've got an otherwise clean traffic abstract and sent in a reasonable sounding letter contesting the infraction, these judges probably aren't going to thoroughly read through every word of it and contrast it with what was alleged in the citation. They don't really care about the city making an extra $173 off your parking ticket -- they just want to get through their citation reviews before lunch. Case dismissed.
So I am not surprised at all by the success of DoNotPay for minor traffic infractions. Most traffic courts are heavily strained by heavy case loads. If you give them a reason to throw your case out so they can go home on time, by all means, they will take it.
I can't speak for lawyers in general or what everyone's motivations would be, but my initial reaction was that it seemed like a somewhat unethical experiment. I assume the client would have agreed or represented themselves, but even there -- legal advice is tricky because it's advice -- it feels unethical to tell a person to rely on something that is very likely going to give them sub-par legal representation.
Sneaking it into a courtroom without the judge's knowledge feels a lot like a PR stunt, and one that might encourage further legal malpractice in the future.
I assume there are other factors at play, I assume many lawyers felt insulted or threatened, but ignoring that, it's not an experiment I personally would have lauded even as a non-lawyer who wishes the legal industry was, well... less of an industry. The goal of automating parts of the legal industry and improving access to representation is a good goal that I agree with. And maybe there are ways where AI can help with that, sure. I'm optimistic, I guess. But this feels to me like a startup company taking advantage of someone who's in legal trouble for a publicity stunt, not like an ethically run experiment with controls and with efforts made to mitigate harm.
Details have been scarce, so maybe there were other safety measures put in place; I could be wrong. But my understanding was that this was planned to be secret representation where the judge didn't know. And I can't think of any faster way to get into trouble with a judge then pulling something like that. Even if the AI was brilliant, it apparently wasn't brilliant enough to counsel its own developers that running experiments on judges is a bad legal strategy.
Run it multiple times with multiple defendants, set up a control group that's receiving remote advice from actual lawyers, mask which group is which to the judges, then ask the judge(s) at the end to rank the cases and see which defendants did best.
That would be a lot more work, but it would also be much higher quality data than what they were trying to do.
And also
> That would be a lot more work, but it would also be much higher quality data
I don’t know much about the field of law, but anecdotally it doesn’t strike me as particularly data driven. So I think, even before introducing any kind of AI, the above would be met with a healthy dose of gatekeeping.
Like the whole sport of referencing prior rulings, based on opinions at a point in time doesn’t seem much different than anecdotes to me.
But I’d love to be proven wrong though.
That said. The consequence of most traffic tickets is increased insurance and a fine. Yes these do have an impact on the accused, but they are the least impactful legal cases, so it would make sense to focus on them as test cases.
Now the question is this. If an AI is doing something illegal like practicing law, how does one sanction an AI?
As far as I'm aware, no LLM has reached sentience and started taking on projects of its own volition. So it's easy - you sanction whoever ran the software for an illegal purpose or whoever marketed and sold the software for an illegal purpose.
https://apnews.com/article/sandy-hook-school-shooting-reming...
https://www.gov.ca.gov/2022/07/12/new-california-law-holds-g...
The second link feeling much closer to direct government action.
I'd prefer that the boilerplate actually be reduced instead, but... I don't have any issue with someone using AI to target tasks that are essentially copy-paste operations anyway. I think this was kind of different.
> If an AI is doing something illegal like practicing law, how does one sanction an AI?
IANAL, but AIs don't have legal personhood, so it would be kind of like trying to sanction a hammer. I don't think that the AI was being threatened with legal action over this stunt, DoNotPay was being threatened.
In an instance where an AI just exists and is Open Source and there is no party at fault beyond the person who decides to download and use it, then as long as that person isn't violating court procedure there's probably no one to sanction? It's likely a bad move, but :shrug:.
But this comes into play with stuff like self-driving as well. The law doesn't think of AI as something that's special. If your AI drives you into the side of the wall, it's the same situation as if your back-up camera didn't beep and you backed into another car. Either the manufacturer is at fault because the tool failed, or you're at fault and you didn't have a reasonable expectation that the tool wouldn't fail or you used it improperly. Or maybe nobody's at fault because everyone (both you and the manufacturer) acted reasonably. In all of those cases, the AI doesn't have any more legal rights or masking of liability than your break pads do, it's not treated as a unique entity -- and using an AI doesn't change a manufacturer's liability around advertising.
That gets slightly more complicated with copyright law surrounding AIs, but even there, it's not that AIs are special entities that have their own legal status that can't own copyright, it's that (currently, we'll see if that precedent holds in the future) US courts rule that using an AI is not a sufficiently creative act to generate copyright protections.
Law is different because the bar has a legally enforced monopoly on doing legal work.
DoNotPay was being threatened. But they weren't practicing law - they were just providing legal tools.
My point is that were in uncharted legal territory. Perhaps ask the AI what it thinks ;)
I don't see how this would decrease DoNotPay's liability.
Regardless of how you feel about the bar, I don't think that changes anything about who they would sanction or why. Having a legal monopoly means they're even less likely to go along with a "the AI did it, not me" explanation than a normal market would be.
I mean, no matter what, they're not sanctioning the AI. They don't recognize the AI as a person, they recognize it as a tool that a person/organization is using to perform an action.
"Personhood" in a legal sense doesn't necessarily mean a natural person. In this case, the company behind it is a person and is practicing law (so no pro se litigant using the company to generate legal arguments). In addition, if you want something entered into court, you need a (natural person) lawyer to do it, who has a binding ethical duty to supervise the work of his or her subordinates. Blindly dumping AI-generated work product into open court is about as clear-cut an ethical violation as you can find.
To your larger point, law firms would love to automate a bunch of paralegal and associate-level work; I've been involved in some earlier efforts to do things like automated deposition analysis, and there's plenty of precedent in the way the legal profession jumped on shepardizing tools to rapidly cite cases. Increased productivity isn't going to be reflected by partners earning any less, after all.
Its not and you don’t.
When a legal person (either a natural person or corporation) is doing something illegal like unauthorized practice of law, you sanction that person. The fact that they use an AI as a key tool in their unauthorized law practice is not particularly significant, legally.
I'm going to sit on that particular hill and see what happens. Even if DoNotPay's AI is not ready to do the job, the idea that AI could one day argue the law by focusing on logic and precedent instead of circumstance and interpretation is exceedingly threatening to a lawyer's career. No offense intended to the lawyers out there, of course. Were I in your shoes, I'd feel a bit fidgity over this, too.
other fields like journalism not so much.
That was my initial response too.
Artists, programmers, musicians, teachers are threatened... but shrug and say "that's the future, what can you do". If lawyers feel "threatened" by AI, they get it shot down.
I suddenly have a newfound respect for lawyers :)
Yet if we think about it, we all have exactly the same tools at our disposal - which is just not playing that game. Difference is, while most professions have got used to rolling with whatever "progressive technology" is foisted on us, lawyers have a long tradition of caution and moderating external pressure to "modernise". I'm not sure Microsoft have much influence in the legal field.
This is a thing that lots of people say about unethical businesses, and I'm a little skeptical about it at this point. A couple of objections I have:
- You have a constitutional right to legal representation when accused of a crime by the US government, and while we don't to abandon people who are suffering now because of some theoretical future fix, we also don't want to normalize the idea that constitutional rights only exist when a private market accommodates them. That's explicitly a bad direction for the country to go.
- Saying "well, this works here and now, and people don't have access to anything better" is in my mind only a really effective argument when we know that the thing here and now actually works. But we don't know that this works, which changes a lot about the equation.
- Is sneaking an AI into a courtroom through an earpiece really a cost-effective accessible strategy for poor people? Nothing about this screams "accessibility" to me.
I think summing up the last two points, if the AI was proven to actually work in a court of law, and was an accessible option, then sure, at that point I think the argument would have a lot more weight. It wouldn't be ideal, it would be a bad state for us to be in because your constitutional rights should not depend on an AI. But I could see a strong argument for using the AI in the meantime.
But that doesn't mean that DoNotPay should do unethical things right now to get to that point. The way that your choice is being phrased is begging the question: it assumes that the AI is the only choice other than no representation, that it does work, and that it will produce better outcomes.
But we don't actually know if the AI does work in a court of law, and DoNotPay's decision was to "move fast and break things"; it was to start releasing it into the wild without knowing what would happen. We don't know if asking people to represent themselves with a secret earpiece is a good legal strategy or if it's accessible. We don't know what happens when something goes wrong. We don't know that this actually is a working solution. But they were putting someone's legal outcome on the line anyway.
I think there's a big difference between making an imperfect solution available to poor people because we don't have anything better to offer, and using poor people as experimental fodder to build an imperfect solution that might not work at all. There's a lot of assumption here that using their AI would be better than representing yourself, and I don't know that's true. A judge is not going to pleased with being used as an experiment. And I've been hearing people say that the AI subpoenaed the officer involved in the ticket? That's not a good legal strategy.
The proper way to build a solution like this is to make sure it works before you start using it on people, and I think it's unethical to give someone bad legal advice and to try and justify it because giving that person bad legal advice might allow the company to help other people down the line. A lot of our laws around legal representation are predicated on the idea that legal advice should be solely focused on the good of the client, and not focused on the lawyer's career, or on someone else the lawyer wants to help, or on what the lawyer will be able to do in the future. Based on what we know about the state of the AI today, it doesn't seem like DoNotPay was thinking solely about the good of the person they were advising. We really don't want the legal industry to be an industry that embraces "the ends justify the means."
Someone using it in an actual courtroom would make a boneheadedly dumb argument or refer to a nonexistent precedent or something. Then maybe the judge gets upset and gives them the harshest punishment or contempt of court or they just lose the case. They may or may not ever get a chance to fix it.
A failure mode of jail time and/or massive fines for your customers doesn't sound all that elegant to me. This isn't a thing to show people cat pictures, I don't think move fast and break things is a good strategy.
Not to say that there aren't some entrenched possibly corrupt and self-serving interests here. But that doesn't mean they don't have a point.
That sounds like a horrible judge, maybe this AI can be used to sniff them out and get rid of them.
Why is the robot not "speech"?
[EDIT] That is, would they be immune from e.g. malpractice if they did this so they "weren't representing" the defendant?
If the book was a generalized "choose your own adventure" where you compose a sensible legal argument from selecting a particular template and filling it in with relevant data - use of the book essentially lets the user find the pre-existing legal advice that is relevant to their situation.
Chatbots as a system are arguably a lot more like the latter than the former - its a tool that someone can use to 'legal advise' themselves.
> Here's how it was supposed to work: The person challenging a speeding ticket would wear smart glasses that both record court proceedings and dictate responses into the defendant's ear from a small speaker.
Also, probably wouldn't matter. The interactive human-ish-like nature might cross the line to being considered as counsel, even if you said it wasn't. See my response to your other comment.
"Speech" would be like publishing a book about self-representation. "Counsel" would be providing advice to a defendant about their specific case. The machine would be in the courtroom advising the defendant on their trial, so that's counsel.
If something that's providing input to a formal legal process (which, let's not forget, means false or inaccurate statements have real and potentially prejudicial repercussions), "makes facts up and exaggerates", then there seems to be no reason they should be talking about taking this anywhere near a courthouse.
This feels a lot like "move fast and break things" being applied - where the people silly enough to use this tool and say whatever it came up with would end up with more serious legal issues. It seems like that only stopped when the founder himself was the one facing the serious legal issues - 'good enough for thee, but not for me'...
I think what many are overlooking is that bad inputs to the legal system can jeopardise someone's position in future irretrievably, with little or no recourse (due to his class action/arbitration waiver). Once someone starts down the road of legal action, there's real consequences if you get it wrong - not only through exposure, but also through prejudicing your own position and making it impossible to take a different route, having previously argued something.
[0] https://twitter.com/SemaforComms/status/1618306993902743555
To avoid the catastrophy that makes it a catastrophically bad idea.
> I assume the real reason it was shot down was out of fear that it would work well. Does anyone else have a better explanation for why there was such a visceral response?
It had already worked badly (subpoenaeing the key adverse witness, who would provide a basically automatic defense win, and one of the most common wins for this kind of case, if they failed to show up.)
We are so close from a new disruptive revolution where a lot of jobs (not just lawyers) will be made obsolete. Possibly similar to inventions like assembly lines, or cars. Such an exciting time to be alive!
The title of the article seems misleading.
A techbro who doesn't appear to be a lawyer or has any understanding of the law wants to use AI so people can defend themselves. It doesn't seem like any of this was done with input from any bar associations. Without seeing the emails and "threats", and ignoring the emotional language it sounds like these people were helping him out:
>"In particular, Browder said one state bar official noted that the unauthorized practice of law is a misdemeanor in some states punishable up to six months in county jail."
Were these emails "angry" or just stating very plainly and with forceful language, that if you do this without the AI having the appropriate qualifications, you are most probably going to jail?
It even sounds like Browder didn't really widely publicise the fact that a case defended by an AI was about to happen.
>As word got out, an uneasy buzz began to swirl among various state bar officials, according to Browder. He says angry letters began to pour in.
Really sounds like these letter writers did him a favour.
1) The legal profession tries to instill a sense of ethics into the lawyers they train, and what DoNotPay was proposing violated that ethics. I don't want to overyhype the legal profession, but many (maybe even most) lawyers really do want to do the right thing, which by their lights means clients get the best representation possible. Which an LLM currently is not, so you get a visceral reaction from people at what they see as discussion of/advocacy of unethical conduct.
2) As a practical matter, it was likely to yield very bad outcomes for all concerned, including professional and legal consequences. Bluntly, DoNotPay was proposing to do something illegal, it really could have resulted in jail time, and possible disbarrment for any lawyers who were involved. For good or ill, judges have immense power to control what goes on in their courtrooms, and the risk of a judge taking offense to this is high. And the higher profile the case (and DoNotPay was offering $1m for a lawyer who would repeat whatever the chatbot said in a Supreme Court case) the higher the stakes. That really could be a career ending mistake for a junior lawyer (plus, almost by definition, anything that reaches the Supreme Court is important; bad representation could throw the result, with potentially terrible consequences for the country).
3) This is not, by any means, the first attempt to try and automate or streamline the provision of basic legal services. And it's a field ripe for such things; simple stuff like many rental agreements, employment contracts, sueing in small claims court, divorce agreements, etc., etc. all seem like they should be able to be generated by filling out a form and pressing a button, not hiring a high priced professional to craft a bespoke document. But over and over, such attempts have failed badly, and lawyers are getting reflexively defensive to anything that looks like that, not because they threaten their jobs, but because they keep being so terrible.
4) And DoNotPay looks a lot like these previous failures. See, eg, https://www.techdirt.com/2023/01/24/the-worlds-first-robot-l... which makes them look less like some cutting edge AI lawyer that will put an army of paralegals out of work, and more like yet another shitty site trying to resell Mechanical Turk at an enormous markup. Now, maybe what they're showing to the public is entirely different than what they want to use in courtrooms, but...it doesn't build confidence.
In many ways I think this is analagous to self driving cars. There is scope for enormous gains here, it's probably inevitable eventually, and it will probably put a lot of delivery drivers, taxi drivers, and truck drivers out of a job when it finally arrives. But right now deciding to hook your new self driving AI up to an unmarked truck and do a secret test in an unnamed US city without warning anyone or getting any licenses or permits is unethical and illegal, and you'd expect a very negative reaction if you announce your plans on Twitter. Especially if your website offers some software that they claim can control an RC car, but when you try it, it keeps steering into walls.
> I assume the real reason it was shot down was out of fear that it would work well
The next lawyer (or AI expert!) I see who thinks it would work well will be the first. There's just an enormous mismatch between the requirements, what chatbots in general seem to be capable of right now, and what DoNotPay in specific seems to be capable of. (Again, see the the Tech Dirt link.)
And again, note that while there's a pretty strong argument that it should be faster, easier, and cheaper to create a rental agreement (or whatever), DoNotPay was making a big deal out of how they wanted to argue in front of the Supreme Court. DoNotPay doesn't seem like they're going to nail the "draft a rental agreement for cheap" any time soon, but maybe they (or someone else) can. But handing a Supreme Court case? Obviously not. Now you might argue (and one certainly hopes!) the whole Supreme Court thing was just an unserious PR stunt, but when a company is making undifferentiated claims that their tech can do something semi-plausible and something entirely impossible, it makes the entire thing look like a scam created by or aimed at people who just don't know any better. Not a good look!
In short: I think the negative reaction is very, very understandable.
That said, if it's such a catastrophically stupid idea, I'm not really sure why it had to be shot down so harshly: seems like that problem would elegantly solve itself. I assume the real reason it was shot down was out of fear that it would work well. Does anyone else have a better explanation for why there was such a visceral response?