An analogy that comes to mind is Excel's autocomplete feature. If you input a company's financials, autocomplete for the future, and publish that as a forecast, you're the one making the forecast. Not Excel.
I do. If I set off a defective firework that careens sideways into your house, I'm liable. I may have a further claim against the manufacturer. But your claim is against me. This is why Mercedes announcing self-driving car liability is such a big deal [1][2]
[1] https://en.wikipedia.org/wiki/Self-driving_car_liability
[2] https://www.kbb.com/car-news/mercedes-well-be-liable-for-sel...
Because a negligence claim requires my estate to prove that you failed to exercise reasonable care.
If the AI is generally reliable, it's reasonable for you to trust it and take your hands off the wheel.
If the AI suddenly goes berserk, it would be hard to prove that you breached your duty of care, and are liable for negligence.
Even though the self-driving system explicitly tells you to keep your hands on the wheel and yells at you if it doesn’t sense them often enough? I don’t think it’s an accident that all of the current systems do a lot of work to make sure you’re aware you’re driving even if the car is helping.
This is building the precedent that manufacturers know whether their systems can handle being unattended, and as soon as they don’t do this they’re liable.
Likewise, suing the person behind the wheel of the self-driving car for negligence should work. Because the "non-driving driver" had the ability to seize the wheel, and should have known that some boneheaded behaviors are a risk with these early AIs.
The owner could then try to go after the company, but they're still ultimately responsible even if the company doesn't cover the cost for them
You are expected to be in control, or be able to be in control at all times. If there is such a possibility, anyways. If there isn't, that means the manufacturer is the one in control.
So yes, I'd say that would change things. IMO.
The user undoubtedly agreed to some T&Cs that mean taking their hands off the wheel even to push something on the touch screen is negligence:
From [1]:
Before enabling Autopilot, the driver first needs to agree to “keep your hands on the steering wheel at all times” and to always “maintain control and responsibility for your vehicle.” Subsequently, every time the driver engages Autopilot, they are shown a visual reminder to “keep your hands on the wheel."
if I was looking at a car with self driving capabilities I'd definitely be looking at Mercedes over pretty much anyone because of that.
Their German website [1] says (translated by Google):
> Our driver assistance and safety systems are aids and do not release you from your responsibility as a driver. Observe the information in the operating instructions and the system limits described there.
It also seems to only work in extremely ideal conditions [2]:
> They must be on one of the nation's 8200 miles of autobahn highway in traffic traveling at no more than 60 km/h (37 mph) in dry conditions and away from tunnels or construction zones.
So basically only works on traffic jams on the autobahn in good weather with no construction. How many accidents are happening in these conditions? This seems like marketing fluff more than anything.
[1] https://www.mercedes-benz.de/passengercars/technology/drive-...
[2] https://www.caranddriver.com/reviews/a39966189/2022-mercedes...
https://www.motortrend.com/news/nhtsa-tesla-autopilot-invest...
No matter the case, though, the law will not throw up its hands, and say there’s no one liable for this tragedy.
LLMs (like GPT) are intended to work despite that ambiguity: they cannot resolve it.
Any time a pattern of text can be used to represent more than one meaning, LLMs lose all of their leverage. Only humans can determine after the fact whether the result is - as GPT's authors have presented it - a "feature" or a "limitation".
It's quite impressive to me how many journalists just take citations at their word. Isn't the point of journalism to verify information?
OpenAI, if you're reading I got a innovative new algorithm for reducing AI harm for fake citations
`if not(link_exists(generated_link)) print(f"[CITATION NOT FOUND]") else print(generated_link)`
I have a more advanced algorithm that uses a small language model to click links that exist and verifies direct quotes. I call this model Regex.
Joking aside, I really do think the problem can be seriously reduced in a simple manner, but also people really need to check their sources.
Thou shalt verify sources, be they generated by the hand of man or of artificial beings.
- John 3:16
For he who uses large language models to autocomplete their claims in greed and sloth shall be damned unto the third generation.
- Exodus 34:7
I'm less sure that's where it should stop. It doesn't seem right to set up a situation introducing powerful transformative tools without any obligations for the people creating it. And as far as I understand it there is indeed a layer of law where manufacturers can be subject to some standards of responsibility.
This might be going against the pro-business attitude of HN, but there's limits to what fine print can accomplish. You can't disclaim away defamation law, you have to convince a court that what you did wasn't defamatory for some articulable reason beyond "I say it isn't" or "I said I wasn't responsible" or some other stock phrase.
If I say, "danmaz74 eats babies," some people are going to believe it and not check that you don't eat babies. That's the problem with saying false things about people. Likewise, if ChatGPT says false things about people, some people will believe those false things - even if they don't publish anything or repeat them. Those false beliefs then have repercussions.
Section 230 doesn't eliminate libel. It just notes that the libel is being done by the person writing the information - ie. the user who posts it rather than the site that is merely hosting the information. It notes that the user is self-publishing the information.
In this case, ChatGPT is publishing this "information" itself. If we hold that ChatGPT isn't responsible for what they publish, we say that no one is liable for this defamation. That's not what Section 230 does. Section 230 just notes who the publisher is - the user who wrote the information. We are determining who is the person doing the action. We could require sites to check everything. We could require your ISP to inspect all packets to make sure they don't contain libel. We could require your electric company to make sure that devices connected to their network aren't being used to send/receive libelous things. We don't because that would be a bit absurd and because the actual fault lies with a human being. In the case of ChatGPT, it seems like we still need to be identifying a culprit.
Let's say that I start auto-generating news articles. I don't check them (I don't even look at them) before they are published on my site. This is the same mechanic as ChatGPT - it's just auto-generated stuff. Therefore, I'm not liable under your definition. It's up to the readers of my site to decide to check this auto-generated stuff.
I agree that ChatGPT can be a useful tool. At the same time, I'm not sure we want to create a system where people can absolve themselves of liability by auto-generating stuff. Even without intent, one can be guilty of libel. Maybe the answer is that ChatGPT should be able to defend itself by showing what sources caused that generation and that the generation was reasonable. For example, a newspaper is expected to carefully check multiple reasonable sources. If ChatGPT can cite multiple reasonable sources (and it has interpreted those sources correctly), then the blame can be those sources. If ChatGPT cannot, then the blame might be on ChatGPT.
As you note, ChatGPT can be useful - and the more useful it gets, the more people will accept its statements as true. I think simply saying "as long as you know what they are," feels like someone trying to say both things - they're incredibly useful and completely trash! It's like when a company tells shareholders "we have a stranglehold on the market" and then tells the government "we have so much competition we're struggling to survive." You can't have both. You can be a novelty tool that you and everyone else thinks of as a joke or you can have a serious tool where libel is a serious issue. The Onion can publish satire because it's a satire site. The NYTimes can't publish something that sounds like a serious NYTimes article and then claim, if sued, that it was actually a satire article and that 100% of readers just read it as serious when it was meant as satire.
If a significant part of our future starts coming from these language models, we will have to ensure a certain standard of care around what they say.
My point is that ChatGPT doesn't "publish" anything; it's not a publication. It's a language model trained on text, which produces text in response to one or more prompts. Using that text is completely up to the user, and it should never be used as a source of facts.
Being able to summarize facts finding specific sources looks like a completely different kind of tool to me, one which would be super useful, but not at all what a LLM in itself is about.
Saying that it is solely the user's misuse of the tool which is the problem minimizes how easy and how damaging the misuse is. Eventually you hit the situation where everyone is making false claims because transitively down the graph a few users were using an LLM that made baseless defaming statements and others cite those publications as evidence for the defamation.
The creators of generative models need to have some skin in the game around the correctness of statements made about real people. Reputational damage cannot be an externality for them. The current AI moment is basking in the glory of how generally right generated text is, when that text is wrong it needs to be their problem too.
Only insofar as they claim that the statements generated by a model should be understood to be factual- and in my experience, not only have I not seen any such claims, I've seen vigorous insistence in the *opposite* direction: that LLM output should *not* be considered factual. OpenAI sticks this disclaimer right underneath their input box: "ChatGPT may produce inaccurate information about people, places, or facts".
> Eventually you hit the situation where everyone is making false claims because transitively down the graph a few users were using an LLM that made baseless defaming statements and others cite those publications as evidence for the defamation.
This is no different from any other gossip chain, except for the fact that the originators used a tool to generate the initial gossip.
This is a legal fig leaf - the reason the AI hype cycle is so high is that it is so often correct. We're seeing it pass medical licensure exams, bar exams, engineering interviews, and so on. Those are all cheered. When it fails we can't just say "you're holding it wrong".
> except for the fact that the originators used a tool to generate the initial gossip.
Scale is a very significant difference. If the tool can generate untold mountains of unique gossip - there is a great need to have some way to control it, and a significant share of responsibility should be on the tool creator. Why should the rest of society have to clean up this mess?
There are significant parallels in this discussion with the argument that guns don't kill people, people kill people statement. Which is also a gross oversimplification of a complex problem.
a better analogy would be tossing a screwdriver off the top of a tall building and then claiming screwdrivers are dangerous because look at how you managed to dangerously distribute one into a windshield when you got your hands on it
Both suggest that this is a real area of concern. Folks have gotten complacent that Section 230 protects platform providers, but it is not clear that the protection extends to outputs from generative models, indeed one of the Supreme Court Justices implied that it would not in oral arguments for one of the recent Section 230 cases.
You can certainly take that output of the tool and publish it but then you become the publisher.
Just like ChatGPT was programmed to drill into a user any time it picks up on being misused? Reminding the user that they are ultimately responsible and liable for their actions, including how they use the output?
From how some make it sound, you would think ChatGPT was giving press conferences.
> oh no but what if it did?
Did it set itself up to do so? No? You found the liable humans.
Every ChatGPT session is personal. You write a prompt and generate output. You're not publishing that output.
> There are four criteria used today in the United States:
> The statement was false, but was claimed as true. > The statement must have been made to a third, previously uninvolved party. > The statement must have been made by the accused party. > The statement caused harm.
> Those who are not classified as public figures are considered private figures. To support a claim for defamation, in most states a private figure need only show negligence by the publisher, a much lower standard than "actual malice."
Laws differ in different states, within the US and without. As you can see here[2], the UK has similar rules to the US (which you might expect but I would say not to presume so easily) in that intent or malice, i.e. mens rea, are rarely part of the equation.
So yes, a machine spitting out what would be defamatory statements from anyone else's mouth are still defamation and would land the publisher in trouble if harm could be ascertained. I'm willing to hazard a bet that most people can see a difference between the kind of thing Google Docs produces when used as a word processor to the kind of thing ChatGPT produces.
[1] https://law.stackexchange.com/questions/130/how-is-defamatio...
[2] https://www.blakemorgan.co.uk/a-brief-guide-to-the-tort-of-d...
Who's claiming the statement is true? My website of random sentences that happens to write something about anything is merely coincidental. With ChatGPT it's less random but no less coincidental. It's clearly and obviously fallible and will give you whatever reasonable sounding answer it can.
Infinite monkeys on infinite typewriters will eventually defame everyone.
a) as others have pointed out, ChatGPT is making what a reasonable person would consider a truth claim
b) it’s not entirely random
c) even if it were random as in your example, you’d have a hard time explaining why it was only a few months and not something a bit more like infinity feels to the average person before the defamatory statements popped out. On the balance of probabilities, you failed to implement true randomness or anything like it, and you’d be liable.
What happens when new models have scraped factually wrong data about a person that has propogated, then someone asks the next chatgpt "tell me about person x" and it spits out the bad learned data?
I this it's just an autocomplete on steroids, a powerful bullshitting machine. The fear I have is that this bullshit is going to pollute the digital world pretty much the same way SEO made it impossible to get good results when using a search engine.
When you form a sentence do you catalog every single word in the english language then pick one before saying a word? Or do you have an understanding that for a given word there are only so many words that could follow?
-
AI is a tool. And it's as much a bullshit generator as a human telling a story they definitely completely remember very vividly... https://www.newyorker.com/science/maria-konnikova/idea-happe...
If you misremember the suspect in a case and go around telling everyone the wrong guy committed a crime, you get sued because you went around telling everyone that, not because you misremembered.
That's how markov chain chat bots (a very old technology) works:
https://stackoverflow.com/questions/5306729/how-do-markov-ch...
https://www.baeldung.com/cs/markov-chain-chatbots
That's not how ChatGPT works though, because of the attention mechanism
GPT is still (partially) probabilistic, and the "it's just autocompleting" refrain stems from this idea that being probabilistic without "higher order intent" means a system is just a bullshit generator.
-
The section of my comment you quoted is not comparing LLMs to Markov chains, it's questioning that notion: Obviously we humans don't consciously evaluate [every single word in our language * each word in the sentence]
So the pool of words that we can consciously speak in a sentence is being defined before we apply higher order intent.
If lacking higher order intent is what makes it "just autocomplete", then we're all just interfaces for autocomplete.
-
Complete this sentence with the scariest thing that comes to mind: "We went to the park and it was fun, but there was a scary..."
The specific sequence "mass hippo attack" probably didn't come to mind even though that'd probably be deadlier than what you thought of.
But that's a pointless observation: After all, what are the odds of a hippo attack happening at the park let alone several? A "mass hippo attack" is so unlikely that you might have already rejected my claim since your scary thing is much more likely.
The point is that you didn't consciously compare "hippo attacks" to whatever you thought of until it was brought up.
And that's because don't often mention hippo attacks in our recollection of going to the park... so the bullshit generator wouldn't surface that for our higher level mind to consider.
-
It turns out just having a probabilistic model of our language is enough to align with higher level thought very often. So often that I challenge the notion that higher level intent drives things. I consider the lower level bullshit generator as running the show, and the higher level self is more like a director who can ask to reshoot the scene, but can't just walk up and act out every role on stage as they please.
We all have bullshit generators that don't care if our higher order self is not racist/misogynistic/etc. and will gladly fill in blanks with hallucinations.
What matters is that our higher order self chooses to reflect and evaluate the pool we surface rather than just blurting out the first thing it surfaces. To me using GPT is no different.
I tried to get ChatGPT to summarize a music video today. (Sugar’s “If I Can’t Change Your Mind”)
I expected it might give a bland summary or something, but that’s not what happened.
It invented entire scenarios that weren’t in the video at all, and invented lyrics not in the song.
That’s pretty harmless, but I can easily see ChatGPT inventing some awful story about a person and that being carried over to a publication and gaining a life of its own.
Basically it’s a souped up urban legend generator, but it’s being offered as a tool to provide search results and content. It’s not just an unreliable narrator - it’s an unreliable narrator being offered as an expert witness.
Often times people use LLMs for generating ideas that don't have a factual basis.
ChatGPT will happily invent gameplay mechanics that are fun. It will generate prompts for convincing concept art for something you haven't built.
If what you're looking for can be answered by Google, sure the business people at Microsoft would rather you use a portal that never lets you leave their site... but that's not interesting.
You could throw darts at a spinning wheel with real names and imagined crimes.
The point is that it doesn't matter what the seed for the false statement is, it's the act of spreading it that's problematic.
You're also muddling a point that I can agree with: Treating ChatGPT as an infallible expert is wrong.
But that applies to so many other things. Even expert witnesses are not infallible.
So I disagree with characterizing hallucinations as the problem, it's the application that's problematic.
Blindly and pasting factual content from ChatGPT is a bad idea, just like blindly taking a single source of information as gospel is a bad idea.
Humans can be just as confidently wrong as LLMs, and a simple adage applies to both: trust but verify.
Trust people who have earned trust (either through qualifications or reputation) and treat everyone else as good faith actors who are quite possibly wrong.
ChatGPT should be treated as a person you just met at a bus stop who is well dressed and well spoken but has just told you that you are both waiting for the elevator to arrive at the bus stop.
I wouldn't expect a random doctor approached at a bus stop to accurately answer a question about medicine anymore than I would ChatGPT by the way. Trusting people based on their qualifications and reputations isn't really a thing.
If a doctor tells you to take medication X there's a reason you take that to a pharmacist rather than a store clerk with a key to a safe or something: verifying is always a great idea, regardless of reputation.
Ahahahahaha. Wow. This is brilliant mate. I'm going to start using it.
An "AI" is a tool.
If ChatGPT makes the same libelous claim in its output to 100,000 people, that doesn't seem materially different from a libelous claim printed in a newspaper with circulation of 100,000.
Microsoft/OpenAI can put down all the legalese disclaimers they want, but if they market ChatGPT/Bing as a tool that provides useful answers, the disclaimers don't protect from libel. By analogy, the NYT can't put a disclaimer in its paper that none of its reporting should be considered truthful in order to protect itself from libel suits -- it just won't work in court. (And yes there are tabloids that print nonsense stories about aliens and Elvis and Brad Pitt, which are for "entertainment purposes only", but the different is the average consumer knows it's a bunch of nonsense, just like The Onion. Parody/fiction is protected.)
So I actually think this is going to be the biggest question/risk by far in terms of commercializing ChatGPT etc. I think the copyright question will be resolved in favor of language models being able to train on anything and everything, but libel is going to be incredibly tricky.
Because the courts don't decide libel on disclaimers, they decide on intent and how a reasonable person interprets things. If they market Bing/ChatGPT as a useful research tool with advertisements showing it giving correct answers etc. then there's a real risk libel suits are going to shut the whole thing down.
On the other hand, to make sure they can operate, they may have to market the thing basically as a toy -- a magic 8 ball, a Mad Libs generator. A disclaimer isn't enough, it means they need to avoid any kind of advertising or product positioning that ever depicts it as giving useful/correct information at all. No more homework help, no more trip planning, no more search tool. Which basically sinks the product. But which is also arguably the best outcome -- that Microsoft/OpenAI shouldn't be permitted to market it as anything but a constantly-lying toy.
And then of course there's the court of public opinion. Even if libel laws are revised to allow for this, if "Microsoft/ChatGPT ruined my life with its lies!" stories start becoming commonplace, it'll just be shut down as a business risk.
But OpenAI at least tacitly suggests that ChatGPT can be used for information retrieval. I know they also say that it can generate false information, but its not hard to see how this could cause problems.
Well yes. But what would you do, if ChatGPT, that is used by 100 mio+ people, consistently would asssociate your name with sex crimes?
Would you shrug it off, because "people know it is just a imperfect tool"?
Throw enough dirt and something will stick, so I would try to get my name cleared ASAP from ChatGPT. But apparently there is no working mechanisms to do so. And this is not acceptable.
If people are tinkering with llms in their basement and it spits out nonsense, no one cares. But ChatGPT is offering an online service for laypersond to use. So if they consistently smear people, then they do have responsibility to stop it.
* If somebody asks ChatGPT to generate some accusation, doesn't check if they're factual, and then publishes them anywhere, then the responsibility lies with somebody.
* If a company directly publishes content from ChatGPT as if it was anything factual or trustworthy, then the responsibility lies with that company. I didn't read the fine print of Bing using it to augment its search engine, but that is definitely a very interesting case.
In the end, ChatGPT and the likes can be incredibly useful tools, as long as you know what they are, ie, very advanced text manipulation tools which generate combinations of words which follow patterns which are common in the training set, and nothing more.