News for hackers. Stuff that matters.
Re:Google pulls AI model after senator says it fabricated as... by jqpabc123 on Monday November 03, 2025 @11:42AM (#45801058)
This is about these tools being blatantly flawed and unreliable.

In legal terms, marketing such a product is called "negligence" or "libel".

Lots of software is flawed and unreliable but this is typically addressed in the terms of service. This may not be possible with AI because the "liability" can extend well beyond just the user.

Re:Google pulls AI model after senator says it fabricated as... by flufluflufluffy on Monday November 03, 2025 @12:05PM (#45801372)
The purpose of the tool is for writing code, it is not for generating factual English sentences.
Re:Google pulls AI model after senator says it fabricated as... by mikkupikku on Monday November 03, 2025 @12:13PM (#45801480)
I do think that might be the only thing they turn out to be any good at, and only then because software is relatively easy to test and iterate on. But does that mean writing code is what the models are "for"? They're marketed as being good for a lot more than coding.
Re:Google pulls AI model after senator says it fabricated as... by jqpabc123 on Monday November 03, 2025 @01:52PM (#45802774)
it is not for generating factual English sentences.

Then the tool should not be doing it --- but it does. And therein is the legal liability.

Re:Google pulls AI model after senator says it fabricated as... by flufluflufluffy on Tuesday November 04, 2025 @06:41AM (#45809828)
The tool did it because the person asked it to. They used the tool the wrong way.

The knives are entering people’s guts. They should not be doing that. The knife companies should be liable for these stabbings.

Re:Google pulls AI model after senator says it fabricated as... by jqpabc123 on Tuesday November 04, 2025 @07:13AM (#45810038)
The tool did it because the person asked it to.

The tool did it because this is what it was designed and trained to do --- at great expense and effort --- but somewahat less than successfully.

You can't have it both ways --- the tool can't be "intelligent" yet too stupid to understand it's own limitations.

If people ask the wrong questions, the "intelligent" response would be, "Sorry, I can't do that".

Maybe the problem here is that this "intelligent" tool is really as dumb as a rock. And it's only a matter of time until lawyers argue this point to a jury in court.

Re:Google pulls AI model after senator says it fabricated as... by jwitthuhn on Monday November 03, 2025 @02:33PM (#45803360)
It is wrong to release something unreliable even while acknowledging it is unreliable? The product performs as advertised. If people want accurate information an LLM is the wrong tool for the job.

From the Gemma 3 readme on huggingface: "Models generate responses based on information they learned from their training datasets, but they are not knowledge bases. They may generate incorrect or outdated factual statements."

Re:Google pulls AI model after senator says it fabricated as... by jqpabc123 on Monday November 03, 2025 @10:08PM (#45807071)
If people want accurate information an LLM is the wrong tool for the job.

So these vendors spent lots of time and money training LLMs to answer questions that people should not ask --- but are allowed and encouraged to.

Nonsensical and unrealistic. I expect the courts will agree and hold the vendors liable.