News for hackers. Stuff that matters.
Re:ChatGPT hit with privacy complaint over defamatory halluc... by crazygringo on Thursday March 20, 2025 @04:27PM (#43428504)
Not really sure what the group expects to achieve with its complaint.

LLM's hallucinate. They just do.

If companies are held liable, then they just... won't make LLM's available in countries where they are held liable.

Is that really the desired outcome? And if it is, it ought to be decided by democratic legislatures, not courts.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by acdha on Thursday March 20, 2025 @06:49PM (#43429956)
Flip that argument around: if your company can’t offer a product without rejecting any responsibility for its reliability, is it really a good product?

That doesn’t mean that companies can’t use LLMs, it only means that they have to market them accurately and use them appropriately. The problem is that companies want to oversell their capabilities because most of the valuable applications need accuracy. Just as we don’t exempt restaurants from hygiene requirements because it’s burdensome, we shouldn’t let the get-rich-quick guys sell something they can’t actually deliver.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by ziddoap on Thursday March 20, 2025 @04:36PM (#43428632)
>LLM's hallucinate. They just do.

Would you be okay if LLMs start responding that you murdered or diddled your kids when responding to "who is crazygringo" queries?

Hallucinating about a cookie recipe or whatever is one thing. But people blindly trust what LLMs spit out. They shouldn't, but they do. This sort of hallucination can cause real damage to real people.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by lxgr on Thursday March 20, 2025 @05:14PM (#43429068)
I'd see it exactly the opposite way: If we don't hold LLM companies accountable for their products at all, we're leaving an important feedback loop open.

There's definitely a sweet spot, but "no consequences whatsoever" is probably too far on the other side of "overwhelming flood of regulations/lawsuits".

Re:ChatGPT hit with privacy complaint over defamatory halluc... by quantified on Thursday March 20, 2025 @06:37PM (#43429846)
Sure is desired. Put yourself in the position of being defamed. Air Canada was forced to honor bogus-low prices described by its chatbot, for example. That doesn't stop Air Canada or anyone else from using LLM chatbots, but (in Canada) clarified that there can be liability.
Re:ChatGPT hit with privacy complaint over defamatory halluc... by mattmanser on Thursday March 20, 2025 @04:36PM (#43428633)
What rubbish.

This is the problem with the internet in a nutshell.

A company that publishes libel and harmful content deserves to be fined.

They used to be, but somehow the whole of social media got a get out of jail free card.

And look what's happened. The collapse of western intellectualism, democracy and liberalism happening before our eyes.

We should not make the mistake of giving that disastrous "I didn't make it, I'm just publishing it" get out of jail card to AI companies.

If they somhow can't fix it, while a certain other AI company can magically fix whinnie the poo references, then they should be prosecuted.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by notavalleyman on Thursday March 20, 2025 @04:47PM (#43428753)
> A company that publishes libel and harmful content

Do you believe that a company who offers an llm to the public, could be said to have 'published' the generative output?

Llms are day dream machines - is it libel if I tell you that I had a dream about you, where you killed a guy? (Has HN just published a harmful lie about you?)

Re:ChatGPT hit with privacy complaint over defamatory halluc... by ziddoap on Thursday March 20, 2025 @04:56PM (#43428875)
>is it libel if I tell you that I had a dream about you, where you killed a guy?

This isn't even closely analogous. I don't know if you could come up with a more bad faith argument.

This was outputting a lie, presented as a fact, to anyone in the world that searched the name.

There is a difference in context (dream vs. fact), difference in scale, difference in expectation (machine outputting what is advertised as accurate information vs. random chatter on a forum where the expectation of accuracy is not a selling point), different methods of redress (chatter can correct you via comment, not so much with an LLM).

Re:ChatGPT hit with privacy complaint over defamatory halluc... by notavalleyman on Thursday March 20, 2025 @05:33PM (#43429248)
> This was outputting a lie, presented as a fact, to anyone in the world that searched the name.

I think you are wrong.

From my understanding, the complainant opened a new chat window and typed "who is forename surname?"

The daydream machine then daydreamed some output text, as is it's function.

Likewise, you can go now to any llm and ask it a specific question like "what is the minimal cheese principle?" (Which I've just made up) And many will daydream a consistent answer for you. As is their function

Re:ChatGPT hit with privacy complaint over defamatory halluc... by ziddoap on Thursday March 20, 2025 @05:44PM (#43429331)
>The daydream machine then daydreamed some output text, as is it's function.

These are not advertised as daydream machines.

They compete on how accurate they they are against various accuracy benchmarks.

The average person who uses them does so with the expectation of accurate results, you know, as they are advertised. Accuracy and speed are pretty much the entire business model.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by notavalleyman on Thursday March 20, 2025 @05:53PM (#43429408)
No, they generally do not compete on accuracy benchmarks afaik.

GitHub/openai/simple-evals is what I checked here, and no, openai do not compete on accuracy benchmarks as far as I can tell. So I'd be interested in seeing what led you to think that, and also what led you to earlier claim that anyone typing in the complainant's name saw the same hallucination.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by ziddoap on Thursday March 20, 2025 @05:57PM (#43429449)
>No, they generally do not compete on accuracy benchmarks afaik.

"Get Answers" is literally at the top of ChatGPTs landing page. You think the average person interprets that to mean "Get inaccurate answers"?

Google "AI benchmark" and almost every result is an assessment of the accuracy of various models. What do you think they compete on? How do you think they measure the improvement of one model to the next?

Here's OpenAI's "Optimizing LLM Accuracy" https://platform.openai.com/docs/guides/optimizing-llm-accur...

Pop this in Google and see the pages of results about accuracy: site:openai.com "accuracy". To claim that they don't optimize for accuracy confirms to me that you are not discussing this in good faith. Perhaps you are just trying to be contrarian or something, I don't know.

>and also what led you to earlier claim that anyone typing in the complainant's name saw the same hallucination.

Well, it says right in the article that different people received the same result.

Why are the goalposts moving? Actually, nevermind, I don't care to continue the conversation.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by notavalleyman on Thursday March 20, 2025 @07:01PM (#43430052)
I think if you take a few moments to read carefully.

You'll see that AI companies, including openai, are generally not competing on accuracy benchmarks.

For example, here are the benchmarks on which open ai seem to be trying to compete.

MMLU: Measuring Massive Multitask Language Understanding,

MATH: Measuring Mathematical Problem Solving With the MATH Dataset,

GPQA: A Graduate-Level Google-Proof Q&A Benchmark,

DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs,

MGSM: Multilingual Grade School Math Benchmark (MGSM), Language Models are Multilingual Chain-of-Thought Reasoners,

HumanEval: Evaluating Large Language Models Trained on Code,

Re:ChatGPT hit with privacy complaint over defamatory halluc... by ziddoap on Thursday March 20, 2025 @07:42PM (#43430326)
I don't know why I'm bothering. But notice how all of these explicitly mention accuracy? And how they are benchmarking the accuracy of the LLM against a known dataset? How accuracy is the primary metric they are evaluated on? Maybe it's because they are trying to improve the accuracy of the models...

First line of the abstract of MMLU: "We propose a new test to measure a text model's __multitask accuracy__."

Fourth line of the abstract of MATH: "To facilitate future research and __increase accuracy__ on MATH"

Second line of GPQA abstract: "We ensure that the questions are high-quality and extremely difficult: experts who have or are pursuing PhDs in the corresponding domains reach __65% accuracy__ [...] while highly skilled non-expert validators only reach __34% accuracy__"

Fifth line of the DROP abstract: "We apply state-of-the-art methods from both the reading comprehension and semantic parsing literature on this dataset and show that the best systems only achieve 32.7% F1 on __our generalized accuracy metric__"

From the MGSM paper: "MGSM __accuracy__ with different model scales."

Models are designed to output accurate information in a reasonable amount of time. That's literally the whole goal. The entire thing. A math-specific model wants to provide accurate math answers. A general model wants to provide accurate answers to general questions. That's the whole point.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by notavalleyman on Friday March 21, 2025 @11:24AM (#43436783)
None of those relate to factual accuracy about a guy in norway
Re:ChatGPT hit with privacy complaint over defamatory halluc... by ziddoap on Friday March 21, 2025 @12:26PM (#43437612)
How much farther can you move the goalposts? We're already almost on another planet.

You ignored almost everything in my original comment and hyper-focused on accuracy. Then, when confronted with the fact that every single example benchmark you provided is a measure of accuracy, you now say "well, it's not a benchmark about a specific person in norway". Obviously not!

The MATH benchmark doesn't ask "what is 2+2", either. Your argument is "well, math-focused models aren't expected to accurately answer 2+2 because it isn't in the MATH benchmark". It's ridiculous.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by mvdtnz on Thursday March 20, 2025 @04:51PM (#43428806)
> Do you believe that a company who offers an llm to the public, could be said to have 'published' the generative output?

Of course they did. What other term could you possibly use for it, when one goes to a website and that website itself hands you content? The content is certainly not user-generated, it's coming from the website.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by notavalleyman on Thursday March 20, 2025 @05:37PM (#43429270)
Maybe a different way of phrasing it would be, if a website embedded a rng generator, and you see the random number "eight", then did the publisher publish the number eight, or did they publish a rng? In my opinion, it's the latter. Similarly if the rng generated the number 666, we wouldn't assume the website is making some kind of biblical commentary. We'd recognise that the rng produced a random generative output, similar to the op situation. So, to impugn the publisher of a random text generator based on the random content .... If chatgpt generates a murder threat, or pro-terorrism content, or otherwise shouts fire in a theater, do you believe openai as the publishers should face arrest?
Re:ChatGPT hit with privacy complaint over defamatory halluc... by lxgr on Thursday March 20, 2025 @06:53PM (#43429995)
The purpose of a system is what it does.

If you publish a "random number generator" that consistently publishes defamatory statements about a person at a much higher frequency than statistically plausible, that's not a random number generator, it's a defamatory statement generator, no matter what you call it.

In court, your intentions might matter just as much as what the system does, but even there, the name you give it is pretty much irrelevant.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by Nullabillity on Thursday March 20, 2025 @06:50PM (#43429970)
The problem with the Alex Jones defense is that they can never quite seem to figure out whether they're just a silly random number generator, or The Revolutionary Future Of Work(tm) that we should be okay with investing trillions of dollars into training and operating.
Re:ChatGPT hit with privacy complaint over defamatory halluc... by mvdtnz on Thursday March 20, 2025 @06:05PM (#43429521)
> do you believe openai as the publishers should face arrest?

Absolutely I do, yes. An LLM is not a random number generator. It is a tool built for the sole purpose of generating content.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by notavalleyman on Thursday March 20, 2025 @07:10PM (#43430113)
It's very shocking to me that you would reply in the affirmative.

I think you're saying that companies who host generative AI web services, ought to be legally liable if the ephemeral generative content is illegal.

In your mind, should AI companies try and engineer protection from this huge legal risk? It seems criminally insane for a company to host an AI if they're going to be legally liable for the ephemeral daydream content. You should be shorting goog, meta and msft at the very least, because I make their models generate illegal content every night before bed

Re:ChatGPT hit with privacy complaint over defamatory halluc... by Nullabillity on Thursday March 20, 2025 @07:28PM (#43430218)
> It seems criminally insane for a company to host an AI

Yes?

Re:ChatGPT hit with privacy complaint over defamatory halluc... by mvdtnz on Thursday March 20, 2025 @08:29PM (#43430631)
Do I think that companies that host and share illegal content should be held liable? Of course I do. How could you possibly feel any other way?

I'm not shorting anything because I'm not a gambler and my opinion on what should be illegal has no basis in what actually is illegal in USA, a country I have never set foot in.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by mvdtnz on Thursday March 20, 2025 @04:50PM (#43428791)
> LLM's hallucinate. They just do.

Then we can establish that they legally can't operate. Sounds like a good outcome to me.

Re:ChatGPT hit with privacy complaint over defamatory halluc... by lxgr on Thursday March 20, 2025 @06:59PM (#43430041)
As many things, in practice there will be a balancing of legal interests.

Different jurisdictions place vastly different value on privacy, absence of defamation against private individuals etc. vs. the right to process and publish information, but none that I know of absolutely prohibit the latter in favor of the former. (Yes, including the EU, despite all the truths and falsehoods programmers believe about the GDPR.)

Re:ChatGPT hit with privacy complaint over defamatory halluc... by rsynnott on Friday March 21, 2025 @06:52AM (#43434111)
> Is that really the desired outcome? And if it is, it ought to be decided by democratic legislatures, not courts.

It would be up to governments to change the laws if they don't want courts doing this on the basis of existing law. Courts work with the laws they have, and in most countries if you (where you is a person or other legal entity, including a company) negligently lie about a normal person, you may be civilly liable (in many places the bar is higher for _public figures_, but that's not relevant in this case). There is no "magic robot did it" get-out clause.