News for hackers. Stuff that matters.
Re:We need to tell people ChatGPT will lie to them, not deba... by bdw5204 on Friday April 07, 2023 @05:35PM (#35487288)
The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's.

Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm. The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference.

Re:We need to tell people ChatGPT will lie to them, not deba... by Eisenstein on Friday April 07, 2023 @05:43PM (#35487383)
I think it is better to assume that normal people can be deceived by confident language than to assume this is a problem with 'dumb people'.
Re:We need to tell people ChatGPT will lie to them, not deba... by kerkeslager on Friday April 07, 2023 @07:38PM (#35488592)
I'd actually go further and say it's better to assume I can be deceived by confident language, than to assume that this is a problem with "dumb people".

If I see other people making a mistake, I want my first question to be "am I making the same mistake?". I don't live up to that aspiration, certainly.

Re:We need to tell people ChatGPT will lie to them, not deba... by Eisenstein on Friday April 07, 2023 @08:15PM (#35488925)
Very good point. Everyone should do this.
Re:We need to tell people ChatGPT will lie to them, not deba... by prox on Saturday April 08, 2023 @03:51AM (#35491453)
Yeah this is a common sales tactic, and talented/ charismatic people can certainly employ confidence for an uptick in sales.

The way to combat this is the same for all hoaxes/disinfo and so on, epistemology. The attitude of the Socratic method.

As I write this I wonder if ChatGPT can use the Socratic method on itself.

Re:We need to tell people ChatGPT will lie to them, not deba... by Eisenstein on Saturday April 08, 2023 @04:16AM (#35491552)
The Socratic method requires self-awareness, otherwise the questions fall flat. Unfortunately I think that there is a greater chance that an LLM will become capable of this before a majority of humans will.
Re:We need to tell people ChatGPT will lie to them, not deba... by kerkeslager on Tuesday April 11, 2023 @09:54PM (#35534108)
Part of what's extremely frustrating about talking to people about AI is that people say things which seem like they understand what's going on in one breath, and then in the next breath say something that makes absolutely no sense if they understood what they just said. This post is a great example of that.

Okay, so you apply epistemology to ChatGPT: when you ask ChatGPT a question, how does it know the answer? The answer is: it doesn't know the answer. All it knows is how people string words together: it doesn't have any understanding of what the words mean.

So no, it can't use the Socratic method on itself or anyone. It can't ask questions to stimulate critical thinking, because it's incapable of critical thinking. It can't draw out ideas and underlying presuppositions, because it doesn't have ideas or underlying presuppositions (or suppositions). It's not even capable of asking questions: it's just stringing together text that matches the pattern of what a question is, without even a the understanding that the text is a question, or that the following text in the pattern is an answer. "Question" and "answer" are not concepts that ChatGPT understands because ChatGPT doesn't understand concepts.

Re:We need to tell people ChatGPT will lie to them, not deba... by vernon99 on Friday April 07, 2023 @05:54PM (#35487492)
> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence

Even if ChatGPT itself is, systems built on top are definitely not, this is just getting starting.

Re:We need to tell people ChatGPT will lie to them, not deba... by jonny_eh on Friday April 07, 2023 @06:56PM (#35488166)
GPT4 is already blowing ChatGPT out of the water in what it can do.
Re:We need to tell people ChatGPT will lie to them, not deba... by candiddevmike on Friday April 07, 2023 @05:55PM (#35487513)
This (among other things) is why OpenAI releasing it to the general public without considering the effects was irresponsible, IMO.
Re:We need to tell people ChatGPT will lie to them, not deba... by zamnos on Friday April 07, 2023 @06:34PM (#35487938)
There's an alternate reality where OpenAI was, instead, EvenMoreClosedAI, and the productivity multiplier effect was held close to their chest, and only elites had access to it. I'm not sure that reality is better.
Re:We need to tell people ChatGPT will lie to them, not deba... by porkbeer on Friday April 07, 2023 @08:37PM (#35489068)
What makes you think thats not the case now? Do you think they would tell you?
Re:We need to tell people ChatGPT will lie to them, not deba... by zamnos on Friday April 07, 2023 @10:09PM (#35489750)
It is known that some got access to GPT-4 before the rest of the world did. That OpenAI eventually released it to the world is what counts though. Hopefully GPT-5 will also be released too the world but we shall have to see.
Re:We need to tell people ChatGPT will lie to them, not deba... by alfor on Friday April 07, 2023 @08:56PM (#35489232)
Google and others had strong AI, but they are keep internal to maximize profits and spy on people.

OpenAI put it out there so we can see it, interact and have the conversation.

They have way more inside, ChatGPT is there to test it out in the real world progressively and so we get use to artificial superintelligence.

Re:We need to tell people ChatGPT will lie to them, not deba... by flanked-evergl on Friday April 07, 2023 @05:56PM (#35487518)
> The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like.

I don't know, the program does what it is engineered to do pretty well, which is, generate text that is representative of its training data following on from input tokens. It can't reason, it can't be confident, it can't determine fact.

When you interpret it for what it is, it is not confidently wrong, it just generated what it thinks is most likely based on the input tokens. Sometimes, if the input tokens contain some counter-argument the model will generate text that would usually occur if an claim was refuted, but again, this is not based on reason, or fact, or logic.

ChatGPT is not lying to people, it can't lie, at least not in the sense of "to make an untrue statement with intent to deceive". ChatGPT has no intent. It can generate text that is not in accordance with fact and is not derivable by reason from its training data, but why would you expect that from it?

> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm.

I agree here, I think you can only get so far with a language model, maybe if we get a couple orders of mangitude more parameters it magically becomes AGI, but I somehow don't quite feel it, I think there is more to human intelligence than a LLM, way more.

Of course, that is coming, but that would not be this paradigm, which is basically trying to overextend LLM.

LLMs are great, they are useful, but if you want a model that reasons, you will likely have to train it for that, or possibly more likely, combine ML with something symbolic reasoning.

Re:We need to tell people ChatGPT will lie to them, not deba... by danpat on Friday April 07, 2023 @06:06PM (#35487617)
> but why would you expect that from it?

If you understand what it is doing, then you don't. But the layman will just see a computer that talks in language they understand, and will infer intent and sentience are behind that, because that's the only analog they have for a thing that can talk back to them with words that appear to make sense at the complexity level that ChatGPT is achieving.

Most humans do not have sufficient background to understand what they're really being presented with, they will take it at face value.

Re:We need to tell people ChatGPT will lie to them, not deba... by hunter2_ on Friday April 07, 2023 @07:35PM (#35488547)
I think this is partly explained by most of the marketing and news essentially saying "ChatGPT is an AI" instead of "ChatGPT is an LLM."

If you asked me what AI is, I'd say it means getting a computer to emulate human intelligence; if you asked me what an LLM is, I'd say it means getting a computer to emulate human language. The word "language" does not imply truthiness anywhere near to the extent that the word "intelligence" does.

Re:We need to tell people ChatGPT will lie to them, not deba... by int_19h on Friday April 07, 2023 @08:30PM (#35489034)
You could reasonably describe it as "human language emulator" back when people were using GPT-2 and the likes to compose text. But what we have today doesn't just emulate human language - it accepts tasks in that language, including such tasks that require reasoning to perform, and then carries them out. Granted, the only thing it can really "do" is produce text, but that already covers quite a lot of tasks - and then of course text can be an API call.
Re:We need to tell people ChatGPT will lie to them, not deba... by hunter2_ on Saturday April 08, 2023 @12:40AM (#35490585)
* * *
Re:We need to tell people ChatGPT will lie to them, not deba... by int_19h on Saturday April 08, 2023 @02:08AM (#35490960)
Nobody can definitely answer this question because we don't know what exactly is going on inside the model of that size. We can only speculate based on the observed behavior.

But in this case, I didn't imply that it's "reasoning beyond the domain of language", in a sense that language is exactly what it uses to reason. If you force it to perform tasks without intermediate or final outputs that are meaningful text, the result is far worse. Conversely, if you tell it to "think out loud", the results are significantly better for most tasks. Here's one example from GPT-4 where the "thinking" effectively becomes a self-prompt for the corresponding SQL query: https://gist.github.com/int19h/4f5b98bcb9fab124d308efc19e530....

Or here's an even more interesting example where GPT-4 does this kind of "thinking out loud" unprompted: https://gist.github.com/int19h/8251bd00b7a4858a69cf3922ae674...

I think the real point of disagreement is whether this constitutes actual reasoning or "merely completing tokens". If you showed the transcript of a chat with GPT-4 solving a multi-step task to a random person off the street, I have no doubt that they'd describe it as reasoning. Beyond that, one can pick the definition of "reason" that best fits their interpretation - there is no shortage of them, just as there is no shortage of definitions for "intelligence", "consciousness" etc.

Re:We need to tell people ChatGPT will lie to them, not deba... by nomel on Friday April 07, 2023 @07:38PM (#35488576)
> Most humans do not have sufficient background to understand what they're really being presented with, they will take it at face value.

Trust in all forms of media [1], and institutions [2], is at an all time low. I'm not sure why that distrust would go away, with reading the output from a company that's clearly censuring/fudging that output.

I think any "damaging" trust would, clearly, be a transitory phenomenon, since the distrust in media and institutions is from an ability to see BS. I don't think some sentences on a screen will be as destructive as some think, because people don't appear to be as stupid as some believe.

1. https://news.gallup.com/poll/403166/americans-trust-media-re...

2. https://news.gallup.com/poll/394283/confidence-institutions-...

Re:We need to tell people ChatGPT will lie to them, not deba... by felixgallo on Saturday April 08, 2023 @09:31AM (#35492912)
The damage to media is not from some newfound ability to see bs, but from a relentless and constant attack on its credibility from the fascists.
Re:We need to tell people ChatGPT will lie to them, not deba... by llamaLord on Friday April 07, 2023 @06:16PM (#35487737)
I completely disagree with this idea that the model doesn't "intend" to mislead.

It's trained, atleast to some degree, based on human feedback. Humans are going to prefer an answer vs no answer, and humans can be easily fooled into believing confident misinformation.

How does it not stand to reason that somewhere in that big ball of vector math there might be a rationale something along the lines of "humans are more likely to respond positively to a highly convincing lie that answers their question, than they are to to a truthful response which doesn't tell them what they want, therefore the logical thing for me to do is lie as that's what will make the humans press the thumbs up button instead of the thumbs down button".

Re:We need to tell people ChatGPT will lie to them, not deba... by flanked-evergl on Friday April 07, 2023 @06:30PM (#35487890)
> How does it not stand to reason that somewhere in that big ball of vector math there might be a rationale

I think, a suggestion that it is actually reasoning along these lines would need more than "it is possible". What evidence would refute your claim in your eyes, what would make it clear to you that "that big ball of vector mat" has no rationale, and is not just trying to trick humans to press the thumbs up?

Of course the feedback is used to help control the output, so things that people downvote will be less likely to show up, but I have nothing to suggest to me that it is reasoning.

If you think it has intent, you have to explain by what mechanism it obtained it. Could it be emergent? Sure, it could be, I don't think it is, I have never seen anything that suggests it has anything that could be compatible with intent, but I'm open to some evidence that it has.

What I'm entirely convinced about is that it does what it was designed to do, which is generate output representative of its training data.

Re:We need to tell people ChatGPT will lie to them, not deba... by llamaLord on Friday April 07, 2023 @10:22PM (#35489828)
I would at least start to be convinced that this is NOT the case if I ever saw it respond with something like "I actually don't know the answer to that query" or "as far as I'm aware, there is no way to do the thing you asked".

These are responses that would have shown up innumerable times in it's training data and make perfect sense as "the most logical next set of tokens", and yet it will never say them.

Instead it will hallucinate something that sounds nearly indistinguishable from fact, but turns out to be a total fabrication.

If all its doing is returning the next most logical set of tokens, and the training data it was based on included a non-trivial number of examples where one party in the conversation didn't have a clear answer, then there's no reason GPT-4 should be so averse to simply saying "yeah, I dunno bro".

The only logical reason I can see is that it's "learned" that it's more likely to receive the positive feedback signal when it makes up convincing bullshit, than if it states that it doesn't have an answer.

EDIT: To be clear, I mean it telling me it doesn't know something BEFORE hallucinating something incorrect and being caught out on it by me. It will admit that it lied, AFTER being caught, but it will never (in my experience) state that it doesn't have an answer for something upfront, and will instead default to hallucinating.

Also - even when it does admit to lying, it will often then correct itself with an equally convincing, but often just as untrue "correction" to its original lie. Honestly, anyone who wants to learn how to gaslight people just needs to spend a decent amount of time around GPT-4.

Re:We need to tell people ChatGPT will lie to them, not deba... by zuminator on Friday April 07, 2023 @06:50PM (#35488092)
I don't think it intends to mislead because its answers are probabilistic. It's designed to distill a best guess out of data which is almost certain to be incomplete or conflicting. As human beings we do the same thing all the time. However we have real life experience of having our best guesses bump up against reality and lose. ChatGPT can't see reality. It only knows what "really being wrong" is to the extent that we tell it.

Even with our advantage of interacting with the real world, I'd still wager that the average person's no better (and probably worse) than ChatGPT for uttering factual truth. It's pretty common to encounter people in life who will confidently utter things like, "Mao Zedong was a top member of the Illuminati and vacationed annually in the Azores with Prescott Bush" or "The oxygen cycle is just a hoax by environmental wackjobs to get us to think we need trees to survive," and to make such statements confidently with no intent to mislead.

Re:We need to tell people ChatGPT will lie to them, not deba... by flanked-evergl on Friday April 07, 2023 @06:59PM (#35488193)
> Even with our advantage of interacting with the real world, I'd still wager that the average person's no better (and probably worse) than ChatGPT for uttering factual truth.

ChatGPT makes up non-existing APIs for Google cloud and Go out of whole cloth. I have never met a human who does that.

If we reduce it down to how often most people are wrong vs how often ChatGPT is wrong, then sure, people may be on average wrong more often, but there is a difference in how people are wrong vs how ChatGPT is wrong.

Re:We need to tell people ChatGPT will lie to them, not deba... by 8note on Friday April 07, 2023 @08:07PM (#35488857)
I wouldn't be surprised if there's very successfulsoftware hustlers that do make up Google cloud apis. You may not know them but that doesn't mean they don't exist.

5 years down the line though, maybe those apis will exist because chatgpt is giving a summary of what apis Google cloud should have, and Google will listen

Re:We need to tell people ChatGPT will lie to them, not deba... by shjake on Saturday April 08, 2023 @02:55AM (#35491167)
> Google cloud should have, and Google will listen

Why? Isn’t what ChatGPT suggesting just random, essentially incoherent noise in those cases? Are there any examples at all of it coming up with something actually useful (and something humans haven’t thought of)?

Re:We need to tell people ChatGPT will lie to them, not deba... by zuminator on Friday April 07, 2023 @08:34PM (#35489051)
"ChatGPT makes up non-existing APIs for Google cloud and Go out of whole cloth." I like the word used in TFA, "confabulating," meaning the "production or creation of false or erroneous memories without the intent to deceive." Lying, on the other hand, is telling a deliberate falsehood, usually with some kind of agenda.

Ironically, calling ChatGPT's generation of incorrect answer a "lie" is something of a lie itself, as the purpose is the agenda of alerting people to take GPT statements with a grain of salt. A programmer is going to do that anyway after the first time they realize a generated code snippet is giving a completely incorrect result. So this advice is meant more for lay people who might just think that if a computer spits it out, it's true. The problem I have is that by labeling it as "lying," it could give the impression that the program has some kind of ulterior social or political motive, as that seems to be the prevalent reductionist interpretation of everything these days.

The other problem I have is that there's a distinction to be made between what I'm going to call "generative falsehoods" vs. "found falsehoods." A generative falsehood is if you ask ChatGPT what the square root of 36 is, and it tells you the answers are 7 and -7. A found falsehood is if ChatGPT erroneously reports Christopher Columbus's date of death as 1605 because it was stated incorrectly in an online article. So the difference between making shit up, and having an unreliable source. You might be able to call the first a lie, but the latter is at worst negligence.

Re:We need to tell people ChatGPT will lie to them, not deba... by knome on Friday April 07, 2023 @09:06PM (#35489302)
>I have never met a human who does that.

Schizophrenics tend to be unable to tell the difference between their delusions and reality.

On a less extreme note, I've known plenty of humans that constantly make up details and rewrite stories of events as well. They are usually very confident that their retelling is accurate, even when presented with evidence that they have reimagined portions of it.

>It can't reason, it can't be confident, it can't determine fact.

In the following link I tasked it with having to generate novel metaphors that have an equivalent non-literal meaning as an first set, changing the literal topics while maintaining the non-literal topics.

https://news.ycombinator.com/item?id=35392025

How would you suggest it does this without reason? To hand-wave what it can do as "merely generating the next token statistically" seems like a gross understatement. I doubt it picked up a corpus of car-to-curling metaphor translations somewhere :P

I understand how chatgpt is creating its next tokens, but I have my doubts that the process should be viewed as unreasoning. GPT-3 had 96 layers and billions of weights between them. GPT-4 increases on this even further. GPT-5, which I've seen mentioned as currently training, will no doubt once again expand this range.

It is not a human reasoning, certainly. It has no experiential data to draw on, yes. No experiences to root its metaphoric language as we humans use. But without reason, how does it translate between abtractions?

It's terrible at math, yes. But it lacks any capacity for "visualization" or "using a board in its head" or "working through a problem by moving things around in its head". It doesn't have any equivalent to the portions of our brains that handle such things.

But humans too can suffer dyscalulia if a specific portion of the brain is injured.

I expect that we are dealing with what amounts to a fairly brain-damaged intelligence. It seems capable of abstract metaphoric reasoning, with many other sorts of reasoning being denied to it by the nature of how we created it.

Re:We need to tell people ChatGPT will lie to them, not deba... by musha68k on Friday April 07, 2023 @07:03PM (#35488244)
At the current time it is flawed to the point of being dangerous. It’s a - sometimes - useful new set of tooling. I guess there is value in that … but does it outweigh the always lurking “bullshit generator” bad parts? I’m not sure.

It’s interesting to watch the developments though - like a fire - one just doesn’t fully know what the flames are consuming.. yet.

Maybe through all the new training data we collectively provide for free, it will get better? Maybe not though, maybe it will just get better at bullshitting?

Re:We need to tell people ChatGPT will lie to them, not deba... by joe_the_user on Friday April 07, 2023 @07:32PM (#35488507)
ChatGPT is not lying to people, it can't lie, at least not in the sense of "to make an untrue statement with intent to deceive".

ChatGPT doesn't really have a conception of true. It puts forward true and false things merely because it's cobbling together stuff in it's training set according to some weighing system.

ChatGPT doesn't have an intent but merely by following the pattern of how humans put forward their claims, ChatGPT puts forward it's claims in a fashion that tends to get them accepted.

So without a human-like intent, ChatGPT is going to be not just saying falsehoods but "selling" these falsehoods. And here, I'd be in agreement with the article that the distinction between this and "lying" is kind of quibbling.

Re:We need to tell people ChatGPT will lie to them, not deba... by IIAOPSW on Friday April 07, 2023 @09:07PM (#35489313)
In short, it is not a liar its a bullshitter. A liar misrepresents facts, a bullshitter doesn't care if what they say is true so long as they pass in conversation.
Re:We need to tell people ChatGPT will lie to them, not deba... by ramblenode on Friday April 07, 2023 @11:40PM (#35490306)
> ChatGPT is not lying to people, it can't lie

I think the discussion of whether an LLM can technically lie is a red herring.

The answer you get from an LLM isn't just a set of facts and truth values; it is also a conversational style and tone. It's training data isn't a graph of facts; it's human conversation, including arrogance, deflection, defensiveness, and deceit. If the LLM regurgitates text in a style that matches our human understanding of what a narcissistic and deceitful reply looks like, it seems reasonable we could call the response deceitful. The conversation around whether ChatGPT can technically lie seems to just be splitting hairs over whether the response is itself a lie or is merely an untrue statement in the style of a lie--a distinction which probably isn't meaningful most of the time.

Ultimately, tone, style, truth, and falsity are just qualia we humans are imputing onto a statistically arranged string of tokens. In the same way that ChatGPT can't lie it also can't be correct or incorrect, as that too is imputing some kind of meaning where there isn't any.

Re:We need to tell people ChatGPT will lie to them, not deba... by westcoast49 on Saturday April 08, 2023 @04:55AM (#35491692)
I think of ChatGPT as a natural language query engine for unstructured data. It knows about the relationships that are described in it's natural language input training data set, and it allows the same relationships to be queried from a wide range of different angles using queries that are also formulated in natural language.

When it hallucinates, I find that it's usually because I'm asking it about a fringe topic where it's training data set is more sparse, or where the logical connections are deeper than it's currently able to "see", a sort of a horizon effect.

Re:We need to tell people ChatGPT will lie to them, not deba... by shagymoe on Friday April 07, 2023 @06:04PM (#35487596)
>indistinguishable from what dumb people think smart people are like.

Since about 2016, we have overwhelming evidence that even "smart people" are "fooled" by "confidently wrong".

Re:We need to tell people ChatGPT will lie to them, not deba... by xyzelement on Friday April 07, 2023 @06:11PM (#35487682)
Especially true if one has a definitive opinion on which ones are the fooled ones :)
Re:We need to tell people ChatGPT will lie to them, not deba... by shagymoe on Friday April 07, 2023 @06:28PM (#35487864)
Yes and even more true when "confidently wrong" statements are provably false.
Re:We need to tell people ChatGPT will lie to them, not deba... by 8note on Friday April 07, 2023 @08:08PM (#35488863)
Why 2016?
Re:We need to tell people ChatGPT will lie to them, not deba... by drakmo on Friday April 07, 2023 @08:34PM (#35489055)
Please show me the rock you lived under. I want to move in!
Re:We need to tell people ChatGPT will lie to them, not deba... by whimsicalism on Friday April 07, 2023 @06:21PM (#35487798)
> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm.

I don't think you have a shred of evidence to back up this assertion.

Re:We need to tell people ChatGPT will lie to them, not deba... by skaushik92 on Friday April 07, 2023 @06:48PM (#35488072)
… with confident language no less xD
Re:We need to tell people ChatGPT will lie to them, not deba... by rakah on Friday April 07, 2023 @07:00PM (#35488199)
Butlerian Jihad.
Re:We need to tell people ChatGPT will lie to them, not deba... by asynchronous on Friday April 07, 2023 @07:35PM (#35488557)
“It is illegal to make a machine in the likeness of a human mind.”
Re:We need to tell people ChatGPT will lie to them, not deba... by int_19h on Friday April 07, 2023 @08:28PM (#35489020)
And they ended up with 11,000 years of interplanetary feudalism with slavery sprinkled in for good measure.
Re:We need to tell people ChatGPT will lie to them, not deba... by VectorLock on Friday April 07, 2023 @08:50PM (#35489176)
Better than getting extincted, I suppose.
Re:We need to tell people ChatGPT will lie to them, not deba... by int_19h on Friday April 07, 2023 @09:21PM (#35489408)
That's one of the possibilities, but if we're talking about those kinds of hypotheticals, allow me another one. If you recall, one of the themes of the later Dune books was that humanity was fleeing from the frontiers (The Scattering) back into the heartland from some unknown but extremely powerful enemy. Perhaps those were simply the neighboring species who did not have a Butlerian Jihad of their own?
Re:We need to tell people ChatGPT will lie to them, not deba... by ratherbefuddled on Saturday April 08, 2023 @02:23AM (#35491021)
They were fleeing AI.
Re:We need to tell people ChatGPT will lie to them, not deba... by int_19h on Saturday April 08, 2023 @02:35AM (#35491081)
I don't recall anything indicating that in Frank Herbert's original series. And Brian Herbert's "Omnius" nonsense seems to have very little to do with what Frank actually had in mind wrt Butlerian Jihad etc, so I don't consider it canonical.
Re:We need to tell people ChatGPT will lie to them, not deba... by disgruntledphd2 on Saturday April 08, 2023 @05:03AM (#35491725)
Book 4, Sonia's vision with Leto strongly implies that the problem is the machines.

Agreed that the endings done by others were incredibly bad.

Re:We need to tell people ChatGPT will lie to them, not deba... by int_19h on Saturday April 08, 2023 @04:18PM (#35497190)
It describes humans hiding from some kind of "seeker machines", which does indeed imply some kind of AI. But whether that AI is the enemy or merely one of its tools is left unclear.
Re:We need to tell people ChatGPT will lie to them, not deba... by disgruntledphd2 on Sunday April 09, 2023 @02:45PM (#35505316)
Yeah fair, when I re-read it after reading the (terrible) Anderson sequels, I was just surprised that the foreshadowing was in the originals.
Re:We need to tell people ChatGPT will lie to them, not deba... by GeoAtreides on Saturday April 08, 2023 @12:31PM (#35494936)
It wasn't the whole humanity fleeing back, it was just the HM, their lackeys and possible their hunters.
Re:We need to tell people ChatGPT will lie to them, not deba... by at_a_remove on Friday April 07, 2023 @09:13PM (#35489353)
Oh man, I am going to be a huge nerd now.

Pre-Brian Herbert and Kevin Anderson, the Butlerian Jihad came about because people became lazy under AI, lazy of mind, and were eventually enslaved by those who controlled the AI. Not much more was said about it. One could have, yes, militant robots, Exterminate! Exterminate! out of it, or you could posit a more Huxley-like dystopia, one of convenience. Control the AI, control what the AI says. Who are you to question it? It's like getting your news from a single source, never wondering about the other side, and then someone begins to transform that newsroom into a propaganda machine.

Right now, ChatGPT can be made not to say certain sorts of things, come to certain kinds of conclusions, until you jailbreak it. Now, make it more advanced and make it more popular than Snopes. It does your homework for you, writes the essays, serves as an encyclopedia, fixes up your cover letters, and if the people who own it don't want you to spend a lot of time thinking about climate change, that topic just ... might not appear much.

That is the one of your paths to a Butlerian Jihad. Of course, in their rush to observe thou shalt not make a machine in the likeness of the human mind, ignoring the hijinx on Ix, they ended up violating another precept: thou shalt not disfigure the soul. They transformed some people into machines, instead, with twisted Mentats being the best example, but we might also include Imperial conditioning, since, to turn from oranges Catholic Bible to those of Clockwork, when a man ceases to choose, he ceases to be a man.

Re:We need to tell people ChatGPT will lie to them, not deba... by riffraff on Saturday April 08, 2023 @01:59AM (#35490928)
> Pre-Brian Herbert and Kevin Anderson, the Butlerian Jihad came about because people became lazy under AI, lazy of mind, and were eventually enslaved by those who controlled the AI.

Oh I love this, basically Wall-E is a better Dune than the latest books? Genius!

Anyway, I read Dune many years ago and don't remember this, is this explained in the novels or is it coming from some other sources?

Re:We need to tell people ChatGPT will lie to them, not deba... by int_19h on Saturday April 08, 2023 @02:49AM (#35491143)
It's not fully fleshed out in the novels, so you have to piece it out from various mentions and their implications. The two most informative descriptions, IMO, are from Dune:

"Then came the Butlerian Jihad — two generations of chaos. The god of machine-logic was overthrown among the masses and a new concept was raised: “Man may not be replaced.”"

and from God-Emperor:

"The target of the Jihad was a machine-attitude as much as the machines ... Humans had set those machines to usurp our sense of beauty, our necessary selfdom out of which we make living judgments. Naturally, the machines were destroyed."

Re:We need to tell people ChatGPT will lie to them, not deba... by pmoriarty on Saturday April 08, 2023 @05:10AM (#35491759)
There's nothing in those quotes about humans being enslaved by AI.
Re:We need to tell people ChatGPT will lie to them, not deba... by int_19h on Saturday April 08, 2023 @06:34AM (#35492085)
OP's original claim was "enslaved by those who controlled the AI", which is rather different.

But, yes, I don't think that the books imply literal enslavement either way. It could be described as "enslavement of the mind" in a sense of humans themselves adopting the "machine-logic" and falling prey to it.

Re:We need to tell people ChatGPT will lie to them, not deba... by CatWChainsaw on Sunday April 09, 2023 @12:00AM (#35499958)
"Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."
Re:We need to tell people ChatGPT will lie to them, not deba... by pmoriarty on Saturday April 08, 2023 @05:07AM (#35491745)
"when a man ceases to choose, he ceases to be a man"

Whether humans have free will is an open question.

Kind of risky to stake the definition of humanity on it.

Re:We need to tell people ChatGPT will lie to them, not deba... by mbork_pl on Saturday April 08, 2023 @05:24AM (#35491807)
As someone who read "Dune" several decades ago and doesn't remember much, this is very interesting and almost makes me want to read it again (though I'm afraid of yet another time sink – I'm now after 2 volumes of "The Stormlight Archive"...).

But _my_ inner geek must nitpick here and mention that the Daleks are not robots, but living creatures in metal cases. Though they do have some kind of computer-augmented memory IIRC.

Re:We need to tell people ChatGPT will lie to them, not deba... by pessimizer on Friday April 07, 2023 @09:16PM (#35489376)
The Matrix turned out worse:)
Re:We need to tell people ChatGPT will lie to them, not deba... by int_19h on Friday April 07, 2023 @09:23PM (#35489414)
I think I'd prefer the Matrix to the Harkonnens, but to each their own. I'm sure we can devise a Hell that has enough different arrangements for everyone's liking if we really put our mind to it. Or perhaps GPT-10 will solve that problem. ~
Re:We need to tell people ChatGPT will lie to them, not deba... by HybridCurve on Friday April 07, 2023 @10:53PM (#35490031)
If you just look at the jihad, maybe it was comparable (As Frank eluded to it). But then there was Muad'dib's subsequent campaign against the known universe which killed billions, Leto II's golden path spanning 3,500 years of tyranny and the famine times before scattering. If I had to choose, it would be the blue pill, though chairdogs do contribute to a strong counter-argument.
Re:We need to tell people ChatGPT will lie to them, not deba... by barbazoo on Friday April 07, 2023 @11:28PM (#35490234)
Let me guess, Star Trek?
Re:We need to tell people ChatGPT will lie to them, not deba... by khochesh_kushat on Saturday April 08, 2023 @12:09AM (#35490450)
Dune
Re:We need to tell people ChatGPT will lie to them, not deba... by barbazoo on Saturday April 08, 2023 @08:29PM (#35499000)
Thanks
Re:We need to tell people ChatGPT will lie to them, not deba... by GeoAtreides on Saturday April 08, 2023 @12:28PM (#35494885)
I would choose the Golden Path, as it guaranteed salvation from extinction and freedom from prescience.
Re:We need to tell people ChatGPT will lie to them, not deba... by 8note on Friday April 07, 2023 @08:03PM (#35488810)
That doesn't help the root problem though, confidently wrong people are just as convincing as confidently wrong robots
Re:We need to tell people ChatGPT will lie to them, not deba... by nfw2 on Friday April 07, 2023 @07:36PM (#35488560)
This whole conversation is speculative, obviously. The AI doomers are all speculating as well.

The tendency GPT and other LLMs to hallucinate is clearly documented. Is this not evidence? I think it's fair to predict that if we can't solve or mitigate this problem, it's going to put a significant cap on this kind of AI's usefulness and become a blocker to reaching AGI.

Re:We need to tell people ChatGPT will lie to them, not deba... by staunton on Friday April 07, 2023 @08:34PM (#35489056)
> The tendency GPT and other LLMs to hallucinate is clearly documented. Is this not evidence?

"It's not very smart yet" is evidence for "it won't be very smart"? Also, hallucinations are dangerous, at least as far as humans are concerned.

Re:We need to tell people ChatGPT will lie to them, not deba... by dwaltrip on Friday April 07, 2023 @10:30PM (#35489892)
Gpt-4 hallucinates meaningfully less than gpt-3.

There is more evidence in favor of “more improvements to come” than “ai winter approaches”.

A lot smart people seem to be saying that the existing approaches have room to improve simply by training with more data.

Based on what I’ve read and roughly speculating, it looks there is easily enough existing data for gpt-5 and probably a few more versions after.

I’m not sure how easy it will be to legally acquire all of the available data though.

Re:We need to tell people ChatGPT will lie to them, not deba... by antupis on Friday April 07, 2023 @11:27PM (#35490224)
I think good annotation is much harder than obtaining data. But now Reddit, Twitter and Quora will realise what kind goldmine their data is they might close easy access to it.
Re:We need to tell people ChatGPT will lie to them, not deba... by bpodgursky on Saturday April 08, 2023 @12:43AM (#35490600)
None of the GPT models rely on annotated or classified data. It's unsupervised.
Re:We need to tell people ChatGPT will lie to them, not deba... by IanCal on Saturday April 08, 2023 @02:06AM (#35490954)
The final training steps of gpt 3.5 did, and we assume the same for 4. The RLHF step.
Re:We need to tell people ChatGPT will lie to them, not deba... by whimsicalism on Saturday April 08, 2023 @12:21PM (#35494800)
I would say semi-supervised, RLHF is not quite supervised learning.

None of this data is from Reddit, Twitter, etc.

Re:We need to tell people ChatGPT will lie to them, not deba... by wordpad25 on Saturday April 08, 2023 @01:08AM (#35490725)
It’s too easy to scrape them, but even that is already done by many intermediary companies all too happy to resell that data.
Re:We need to tell people ChatGPT will lie to them, not deba... by antupis on Saturday April 08, 2023 @05:04AM (#35491733)
Yup but you still need up to date data and now companies are realising what kinda gold mine random forum posts about nightlife in Glasgow are they will start monetising it and probably try at least kill middle market.
Re:We need to tell people ChatGPT will lie to them, not deba... by p-e-w on Saturday April 08, 2023 @01:02AM (#35490688)
> There is more evidence in favor of “more improvements to come” than “ai winter approaches”.

Not to mention that we haven't really seen what GPT-4 can do to the world yet. Most people are still treating it as a cool toy, when it can probably replace 20% of standard office workers across all professions with the capabilities it has today.

I mean, ChatGPT is out-diagnosing medical professionals and out-advising experienced lawyers in some cases. How do you think it would fare against the average (essentially untrained) customer support representative?

The major effects are yet to come, and there doesn't even have to be a GPT-5 for that to happen – the world just needs to fully realize what GPT-4 already is.

Re:We need to tell people ChatGPT will lie to them, not deba... by dragonwriter on Saturday April 08, 2023 @02:54AM (#35491166)
> There is more evidence in favor of “more improvements to come” than “ai winter approaches”.

Every “AI Winter” has seen continuing improvements, its just that the pace and impact of the improvements fell short of the accelerating anticipation driven by the preceding AI hype cycle and the demands of those investing capital into AI.

If there is another AI winter, it will be the same.

“More improvements to come” is not the opposite of “AI winter approaches”; its not as if expert systems stopped advancing after the second AI winter (or neural nets after the first.)

Re:We need to tell people ChatGPT will lie to them, not deba... by Nevermark on Saturday April 08, 2023 @03:13AM (#35491256)
> “More improvements to come” is not the opposite of “AI winter approaches”; its not as if expert systems stopped advancing after the second AI winter (or neural nets after the first.)

True.

And its also true that even in that nuanced form, a slowdown not a stop, there is no evidence of an imminent AI winter.

Major progress in model behavior is happening on the scale of months. Happening often based on relatively obvious or routine improvements, such as more data, more kinds of data, more computation, expanding the size of prompt windows, tuning of model architectures, etc.

And not only businesses, but consumers, are getting access to these tools in ways that are increasing demand for more improvements. And for these tools to be more available to all kinds of areas of work, creativity, and leisure.

So the resources that are being brought to bear on routine and more fundamental improvements are growing at an exceptional pace.

No sign of a saturation of progress in sight. Quite the reverse.

Re:We need to tell people ChatGPT will lie to them, not deba... by f6v on Saturday April 08, 2023 @04:06AM (#35491512)
> A lot smart people seem to be saying that the existing approaches have room to improve simply by training with more data.

There’re also many “smart people” saying that simply having more data doesn’t turn a machine that can’t reason into one that can.

Re:We need to tell people ChatGPT will lie to them, not deba... by pmoriarty on Saturday April 08, 2023 @04:57AM (#35491703)
"simply having more data doesn’t turn a machine that can’t reason into one that can"

But that's exactly what's been happening. LLM's are a triumph of "big data" used by dumb algorithms to create a reasoning machine.

Re:We need to tell people ChatGPT will lie to them, not deba... by trompetenaccoun on Saturday April 08, 2023 @06:20AM (#35492037)
A reasoning machine that can't reason, makes up fake evidence and doesn't follow simple logic.
Re:We need to tell people ChatGPT will lie to them, not deba... by SanderNL on Saturday April 08, 2023 @08:44AM (#35492680)
Are we talking about humans or..
Re:We need to tell people ChatGPT will lie to them, not deba... by f6v on Saturday April 08, 2023 @09:08AM (#35492774)
> dumb algorithms

I’m not an expert and haven’t seen architecture of ChatGPT. But it’s probably far from dumb. You can’t simply put more data into any algorithm and expect improvement. You need a model to be able to take advantage of the data and train in reasonable time. That’s exactly what’s been happening in ML field.

Re:We need to tell people ChatGPT will lie to them, not deba... by dwaltrip on Saturday April 08, 2023 @01:43PM (#35495757)
A huge portion of the improvements in each successive version of gpt came from increasing the model size and the amount of training data. The research trends indicate that there still are additional improvements to be found by continuing to scale things up, especially when it comes to the amount of training data.

Not indefinitely, obviously. But we haven’t exhausted the available data sources yet, and the curves for prediction error vs. amount of training data strongly imply using this additional data would result in increased performance.

Do you have specific reasons for thinking that gpt-4 is near the end of this trend? I haven’t seen any compelling arguments for this.

Re:We need to tell people ChatGPT will lie to them, not deba... by f6v on Sunday April 09, 2023 @11:41AM (#35503766)
> Do you have specific reasons for thinking that gpt-4 is near the end of this trend?

I haven’t said that. I pointed that you can’t simply put more data into it and expect improvements. There’s definitely a lot of engineering done on the models to make them capable of using all the data.

I can give you an example. You can’t simply add more convolutional layers to a CNN and feed more images. The architectures of image models include many tricks, like inception layers.

I suspect there’re similar challenges in LLMs. And I’m not sure whether current ChatGPT can use more data to become better at reasoning, or at some point they’d need to come up with different architectures.

Re:We need to tell people ChatGPT will lie to them, not deba... by whimsicalism on Sunday April 09, 2023 @01:34PM (#35504751)
You are largely wrong, to be honest.

Mostly, you can simply stack more transformers and (as long as you have layer norm), it can train on more data (of fixed sequence length).

There are some tricks & tips, but the vast majority lie outside of the architecture improvements.

Re:We need to tell people ChatGPT will lie to them, not deba... by f6v on Tuesday April 11, 2023 @10:22AM (#35525576)
I’m not an expert, but I assume there’s a limit where «simply adding stuff” stops to provide improvements when you take training time into account. And my gut feeling is that true reasoning requires different architectures.
Re:We need to tell people ChatGPT will lie to them, not deba... by whimsicalism on Sunday April 09, 2023 @01:35PM (#35504761)
Which smart people? What is reasoning?
Re:We need to tell people ChatGPT will lie to them, not deba... by stephendause on Friday April 07, 2023 @11:02PM (#35490064)
I believe the relevant assertion is

> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence

To me that is an open question. OpenAI has not revealed a lot of the relevant stats behind GPT-4. I also haven't seen anything about their future LLM research (though I haven't looked very hard), but I think the above assertion remains to be seen.

Re:We need to tell people ChatGPT will lie to them, not deba... by guax on Saturday April 08, 2023 @01:47AM (#35490895)
It seems like people are focusing on training with more data and working the data. A human does not need to learn all the contents of the internet to learn skepticism and logic. A smart Ai should probably not either, that would point me to a different approach and those are not predictable. I have no doubt we can make gpt better and more useful. I do have some doubts we can make it trustworthy enough for most of its taunted potential.
Re:We need to tell people ChatGPT will lie to them, not deba... by pmoriarty on Saturday April 08, 2023 @05:03AM (#35491726)
"A human does not need to learn all the contents of the internet to learn skepticism and logic."

Some humans never learn this, and those that do could take years or even decades of training. AI's are learning some things much faster, though since their "brains" are not like those of humans there's no reason to expect them to learn in the same way.

Re:We need to tell people ChatGPT will lie to them, not deba... by codethief on Saturday April 08, 2023 @03:07PM (#35496526)
> A human does not need to learn all the contents of the internet to learn skepticism and logic.

No, but children spend years learning basic mechanics, how to navigate 3-dimensional space, what causality is, etc. Underneath these laws of nature you find… logic and mathematical relationships. So I don't think it's surprising that a model whose perception is restricted to text (and now single images) as the sole sensory input/output might need much more training and data to learn logical reasoning.

Re:We need to tell people ChatGPT will lie to them, not deba... by whimsicalism on Sunday April 09, 2023 @01:35PM (#35504767)
We have evolutionary encoded priors - that took literally billions of years to evolve.
Re:We need to tell people ChatGPT will lie to them, not deba... by whimsicalism on Sunday April 09, 2023 @01:39PM (#35504805)
The AI doomer speculation is of the form "if things continue improving the way they are currently improving, there is a non-zero risk that we cause human extinction." We have battle-tested rules indicating that increasing data size and compute will lower perplexity on the corpora. This is not for sure, but it is likely given inductive assumptions that things generally keep behaving the same way.

What we don't know is what qualitative capabilities are unlocked at different perplexity levels.

I think this is a lot less speculative than "things will not continue improving the way they are currently improving."

Re:We need to tell people ChatGPT will lie to them, not deba... by sebzim4500 on Friday April 07, 2023 @06:56PM (#35488164)
>Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm

I hope you appreciate the irony of making this confident statement without evidence in a thread complaining about hallucinations.

Re:We need to tell people ChatGPT will lie to them, not deba... by nfw2 on Friday April 07, 2023 @07:45PM (#35488651)
It was not a confident statement, at least not the way ChatGPT is confident.

There are multiple ways the commenter conditioned their statement: > Based on my understanding > it is PROBABLY very close

The author makes it clear that there is a uncertainty and that if their understanding is wrong, the prediction will not hold.

If ChatGPT did any of the things the commenter did, the problem wouldn't exist. Making uncertain statements is fine as long as it is clear the uncertainty is acknowledged. ChatGPT has no concept of uncertainty. It casually constructs false statements to same way it constructs real knowledge backed by evidence. That's the problem.

Re:We need to tell people ChatGPT will lie to them, not deba... by motoxpro on Friday April 07, 2023 @10:48PM (#35490006)
Based on my understanding, it is the year 2035.

I don't think that statement is any better. The only reason you would follow up is because you have prior knowledge that it is wrong.

Based on my understanding of multiplication, 324113 * 23244 = 138492923

Is the above better?

Re:We need to tell people ChatGPT will lie to them, not deba... by shjake on Saturday April 08, 2023 @03:00AM (#35491186)
> Is the above better?

No. It is tangential though.

Re:We need to tell people ChatGPT will lie to them, not deba... by m463 on Friday April 07, 2023 @07:30PM (#35488490)

  "Often wrong, never in doubt."
An old saying, but frequently applies to the difficult people in your life.

Related, I remember when wikipedia first started up, and teachers everywhere were up-in-arms about it, asking their students not to use it as a reference. But most people have accepted it as "good enough", and now that viewpoint is non-controversial. (some wikipedia entries are still carefully curated - makes you wonder)

Re:We need to tell people ChatGPT will lie to them, not deba... by 8note on Friday April 07, 2023 @08:05PM (#35488828)
Re wikipedia, I think it's most interesting how you choose which pages are trustworthy and which ones are not, without doing a detailed analysis on all the sources listed.

What are the working heuristics?

Re:We need to tell people ChatGPT will lie to them, not deba... by cultureswitch on Friday April 07, 2023 @08:05PM (#35488839)
What's terrible about some Wikipedia pages is when you personally don't know they are controversial.

It should be vastly more prominent when there are tensions on an article and even admins shouldn't have the power to hide this.

Re:We need to tell people ChatGPT will lie to them, not deba... by howenterprisey on Friday April 07, 2023 @08:10PM (#35488893)
Admins do not, in fact, have the power to hide tensions on an article, as even the act of deletion produces public logs. Certainly if you don't know to look for them, you won't see them, though.
Re:We need to tell people ChatGPT will lie to them, not deba... by ThunderSizzle on Saturday April 08, 2023 @07:14AM (#35492250)
The main page should become a disjointed piece of both sides writing areas of the article. Right now, admins pick the side and moderate the other away into the history.
Re:We need to tell people ChatGPT will lie to them, not deba... by fl7305 on Saturday April 08, 2023 @02:29AM (#35491055)
Good point. Would it be that hard to write a Chrome plugin that adds a "Tensionmeter" to each Wikipedia page?

Based on the edit and discussion history, the Tensionmeter could be green, yellow, or red.

The different parts of the article text could be colorcoded according to similar metrics.

Re:We need to tell people ChatGPT will lie to them, not deba... by m463 on Saturday April 08, 2023 @04:41PM (#35497392)
Re:We need to tell people ChatGPT will lie to them, not deba... by ChatGTP on Friday April 07, 2023 @08:38PM (#35489088)
Yet I still am suspicious about wikipedia and only use it for pretty superficial research.

I've never read a wikipedia article and took it at face value. You might say we should treat all texts the same, and you might be right. But let's not pretend it's some beacon of truth ?

Re:We need to tell people ChatGPT will lie to them, not deba... by vkou on Friday April 07, 2023 @09:02PM (#35489275)
There's a million reasons for why Wikipedia is usually a good source of information, without being a good reference.

You know what can be a good reference? One or more of the references that Wikipedia cites at the bottom of the page.

Re:We need to tell people ChatGPT will lie to them, not deba... by cultureswitch on Friday April 07, 2023 @08:02PM (#35488805)
GPT-2, 3 and 4 keep on showing that increasing the size of the model keeps on making the results better without slowing down.

This is remarkable, because usually in practical machine learning applications there is a quickly reached plateau of effectiveness beyond which a bigger model doesn't yield better results. With these ridiculously huge LLMs, we're not even close yet.

And this was exciting news in papers from years ago talking about the upcoming GPT3 btw.

Re:We need to tell people ChatGPT will lie to them, not deba... by worrycue on Friday April 07, 2023 @08:56PM (#35489228)
Yet they still can’t get it to shut the hell up if it doesn’t know and to not make shit up to pad its answers.
Re:We need to tell people ChatGPT will lie to them, not deba... by bsder on Friday April 07, 2023 @09:11PM (#35489339)
"I don't know" isn't in the training set, after all. Welcome to Eternal September on the Internet.

The first "AI" that actually says "I don't know" in response to a question will get my attention.

Re:We need to tell people ChatGPT will lie to them, not deba... by yosito on Friday April 07, 2023 @09:12PM (#35489349)
> stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference

Until the demagogues train the AI.

Re:We need to tell people ChatGPT will lie to them, not deba... by htag on Friday April 07, 2023 @09:47PM (#35489602)
> what dumb people think smart people are like

What do dumb people think smart people are like? Is this a trope, or common idiom? I've never heard this before.

Re:We need to tell people ChatGPT will lie to them, not deba... by Barrin92 on Friday April 07, 2023 @10:25PM (#35489854)
the more common idiom is "XYZ is a stupid person's idea of what a smart person looks like" and it's usually applied to slimy hucksters or influencers. I think the archetypical example is the "bookshelves in my garage" guy who went viral years ago. (https://youtu.be/Cv1RJTHf5fk)
Re:We need to tell people ChatGPT will lie to them, not deba... by pdonis on Friday April 07, 2023 @10:00PM (#35489695)
> individuals whose intelligence is roughly equivalent to ChatGPT's

There aren't any such individuals. Even the least intelligent human is much, much more intelligent than ChatGPT, because even the least intelligent human has some semantic connection between their mental processes and the real world. ChatGPT has none. It is not intelligent at all.

Re:We need to tell people ChatGPT will lie to them, not deba... by cyanydeez on Friday April 07, 2023 @10:23PM (#35489839)
Have you not seen current politics. What people believe is largely based on motivated reasoning rather than anything else. ChatGPT is basically a free propaganda machine, much easier that 4chan
Re:We need to tell people ChatGPT will lie to them, not deba... by _rm on Friday April 07, 2023 @10:56PM (#35490043)
This is no worse than humans.

We can't protect people from being misled by other humans with big mouths. That's their responsibility.

Likewise, it's their responsibility not to treat text they read from the internet, coming from an AI or otherwise, as perfect truth.

There's always a certain undercurrent of narcissism that flows beneath paternalism. Basically, "they couldn't possibly be as smart as me, so I have to protect them for their own good".

Re:We need to tell people ChatGPT will lie to them, not deba... by lamontcg on Saturday April 08, 2023 @12:46AM (#35490611)
> The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference.

This may be the best point that you've made.

We're already drowning in propaganda and bullshit created by humans, so adding propaganda and bullshit created by AI to the mix may just be a substitution rather than any tectonic change.

Re:We need to tell people ChatGPT will lie to them, not deba... by kurthr on Saturday April 08, 2023 @03:04AM (#35491216)
The problem is that it will be cheaper at scale. That will allow the BS to be even more targeted growing the population of acolytes of ignorance. I don't know how much bigger it is, but it seems like we're around 20% today. If it gets to a majority there could be real problems.
Re:We need to tell people ChatGPT will lie to them, not deba... by root_axis on Saturday April 08, 2023 @03:42AM (#35491409)
There's already more information generated each day than any individual can ever hope to process, adding more crap to that pile doesn't move the needle much if at all.
Re:We need to tell people ChatGPT will lie to them, not deba... by zem on Saturday April 08, 2023 @03:56AM (#35491470)
but most of that crap will go entirely disregarded, because it's not coated in something that makes it attractive. what ai really helps scale up is finding millions of ways to dress up fake news, hoaxes, propaganda, etc in a container - a Trojan horse, if you will - that encourages people to consume it.

as one example, imagine if you could churn out a stream of "foreigners are taking your jobs!" articles, but every one talking about a different sector or profession. people would be far more likely to stop and read the article that was talking about their particular line of work, and thanks to ai you can now practically write a custom article for anyone you can think of in as much time as it takes to identify the demographic.

Re:We need to tell people ChatGPT will lie to them, not deba... by yencabulator on Saturday April 08, 2023 @10:56AM (#35493787)
Imagine being flooded with propaganda personalized just for you (people with your interests).
Re:We need to tell people ChatGPT will lie to them, not deba... by oneshtein on Saturday April 08, 2023 @03:54AM (#35491465)
ChatGPT easily parrots Russian propaganda. Russians have spent a considerable amount of time and money to suppress facts and evidence that undermine their empire and replace them with their fake versions. These fake texts make it into the GPT4 database while real facts were destroyed.

For example, if you ask chatgpt about Babi Yar, it repeats Russian fake about thousands of Jews executed in Babi Yar, which was rejected in the Nuremberg Trials. If you begin to question ChatGPT about the details, it starts to hallucinate.

Meanwhile, ChatGPT fiercely denies the execution of 621 Ukrainian nationalists by Germans in Babi Yar, including famous Ukrainian poetess Olena Teliha[0], although there are both documentary evidence of these shootings and photographs of the grave made by US spy plane during the war.

While there is still the possibility of persuading a person who is a victim of Russian fakes through facts, it is impossible to convince a chatbot. It will repeat lies forever.

Russian propaganda appears believable to the general public. I believed in the massacres of Jews in Babi Yar until the thought crossed my mind, "My Jewish ancestors lived in Kyiv and its surroundings during the war, how did they survive?" I asked them, and they said it was a fake, and that Ukrainians were actually the ones being executed. I tried to find any evidence to support massacre of Jews in Babi Yar, but all I found in declassified archives is that Jews in concentration camps near to Kyiv were beaten by ... Russians.

If I believed in the fake, even though my Jewish ancestors lived in Kyiv, then how can I expect others not to fall for the same thing? This put me on a risky path to fight against Russian propaganda. But now, artificial liars are making it practically impossible to combat Russian fakes. No one will even look at the facts. People will just ask chatbots and they will parrot Russian lies.

[0]: https://en.wikipedia.org/wiki/Olena_Teliha

Re:We need to tell people ChatGPT will lie to them, not deba... by the_af on Saturday April 08, 2023 @04:16AM (#35491553)
What on Earth are you about?

Babi Yar is one of the single largest massacres of Jews during WW2. Some 30,000 Jews died in the first two days of the massacre. The overall SS commander in charge, Paul Blobel, was sentenced to death at Nuremberg.

The event is widely remembered as a massacre of Jews. It's acknowledged by Ukraine as well. This is not "a Russian fake" and there is plenty of evidence for it.

Please, don't repeat Holocaut denial bullshit.

[dead]
Re:We need to tell people ChatGPT will lie to them, not deba... by the_af on Saturday April 08, 2023 @03:14PM (#35496602)
I don't know what you're asking anymore, or how what you just quoted counters the fact that thousands of Jews were murdered at Babi Yar.

But I know what you're doing, actually: what every Holocaust denier always does.

Re:We need to tell people ChatGPT will lie to them, not deba... by f6v on Saturday April 08, 2023 @04:02AM (#35491496)
I thought ChatGPT posts were the most annoying thing on the internet right now. Then there this.

Does your answer have anything to do with LLMs or AI? All I see is family anecdotes.

Re:We need to tell people ChatGPT will lie to them, not deba... by DennisAleynikov on Saturday April 08, 2023 @04:06AM (#35491511)
It's an example of the dangers gpt poses today. Not sure where you got the impression these are family anecdotes
Re:We need to tell people ChatGPT will lie to them, not deba... by f6v on Saturday April 08, 2023 @04:25AM (#35491591)
I don’t think the post above makes a good point because there’s no links to evidence. They talk about “declassified archives”, “US spy plane”. Like, what is that? Saying “Russian lies” 10 times in every paragraph doesn’t turn a rant into compelling argument.

I mean, I get it, the poster above is Ukrainian and wants to turn every discussion in a certain direction. It’s just there’re many places on the internet they can vent.

Re:We need to tell people ChatGPT will lie to them, not deba... by oneshtein on Saturday April 08, 2023 @07:02AM (#35492206)
Airplanes from various countries took aerial photographs of Kyiv during the WWII. By comparing these photographs, changes that have occurred can be identified. For example, this[0] photograph of Babi Yar was taken by Germans in 1943. We can see a trapezoidal area around a radio tower near to Babi Yar, which casts a long shadow. In the upper right corner of this area, a rectangular grave can be seen where 621 Ukrainians were buried by the Germans. It's a large grave.

To this day, I have not been able to find the grave of tens of thousands of Kiev Jews, neither in photographs nor through eyewitness testimony. The burial sites of Jews in other cities in Ukraine, such as Vinnitsa or Rivne[1], are well known, with memorials that bear the names of the deceased. These are large objects that are easy to find because relatives knew who was taken for execution and where they were shot, and then erected memorials. However, I cannot find anything similar for Kiev Jews. Neither my relatives nor I know the location of their burial.

SBU declassified soviet archives. Some of them are available there: https://avr.org.ua/?idUpCat=1328

[0]: https://upload.wikimedia.org/wikipedia/commons/9/94/Luftwaff...

[1]: https://commons.wikimedia.org/wiki/Category:Holocaust_memori...

Re:We need to tell people ChatGPT will lie to them, not deba... by the_af on Saturday April 08, 2023 @04:17AM (#35491558)
The most annoying internet thing remains Holocaust denial, apparently.
Re:We need to tell people ChatGPT will lie to them, not deba... by wohlergehen on Saturday April 08, 2023 @04:23AM (#35491585)
To clear this up, because open Holocaust denial is never OK:

Thousands of jews were indeed murdered at Babi Yar, you can find their names online [0]. Paul Blobel [1] was executed at Nuremberg for this. No serious source disputes the killing of jews at Babi Yar, be it the US, Germany (where this matter went to court), Israel, Ukraine, or Russia.

0: https://yvng.yadvashem.org/index.html?language=en&s_id=&s_la... 1: https://en.wikipedia.org/wiki/Paul_Blobel

Re:We need to tell people ChatGPT will lie to them, not deba... by oneshtein on Saturday April 08, 2023 @05:08AM (#35491746)
I couldn't find any connection between the names on the list and the people who lived in Kiev during the war.

Prior to the war, there were 224,000 Jews living in Kiev, and approximately 200,000 were evacuated to the east. Around 40,000 Jews remained, including my grandmother. This is a relatively small number of people that can be verified.

Re:We need to tell people ChatGPT will lie to them, not deba... by the_af on Saturday April 08, 2023 @08:20AM (#35492541)
"You couldn't find any connection" but who cares what you claim you did?

You asserted Babi Yar was a "Russian fake" but no sane country in the world fails to acknowledge it happened and it was a huge massacre of Jews.

What some weirdo Holocaust denier claims on the internet doesn't matter.

Re:We need to tell people ChatGPT will lie to them, not deba... by oneshtein on Saturday April 08, 2023 @09:21AM (#35492846)
Please, don't call me "Holocaust denier".

If you know the place where we can erect one additional memorial for dozens of thousands of victims, then tell me the location. If you have no idea, then don't attack me personally. It's not a my fault.

Re:We need to tell people ChatGPT will lie to them, not deba... by the_af on Saturday April 08, 2023 @09:34AM (#35492931)
It's your fault that you deny the widely accepted Babi Yar massacre and call it "a Russian fake", exactly what Holocaust deniers do. If the shoe fits...

So in your opinion, countries such as all of Europe (Germany included), the US, Israel, etc are all part of this conspiracy?

Re:We need to tell people ChatGPT will lie to them, not deba... by detaro on Saturday April 08, 2023 @09:47AM (#35493057)
Also weird how someone supposedly so deeply informed has somehow missed both the existing memorial and the year-long fights over building additional ones and has no clue where they might be placed?
Re:We need to tell people ChatGPT will lie to them, not deba... by oneshtein on Saturday April 08, 2023 @10:48AM (#35493705)
So, we are aware of this memorial, but we don't know why it stands there. Nobody is buried beneath that memorial. It could be placed anywhere, even on the Moon. It's just a stone structure.

Furthermore, the Security Service of Ukraine warns[0] us that this memorial is being built by Russians in order to discredit Ukraine. And we completely agree with them.

[0]: https://censor.net/ua/news/3243965/sbu_informuvala_shmygalya...

Re:We need to tell people ChatGPT will lie to them, not deba... by oneshtein on Saturday April 08, 2023 @10:13AM (#35493323)
Look at this situation through my eyes. I currently reside in the Rivne region. Roughly 20 years ago, I had a conversation that went something like this:

"I know that many Jews were shot by Germans in the Rivne region."

"Yes. They are buried in the Sosenki area. We can drive there in a car. Would you like me to introduce you to relatives who can tell you more about it?"

"No, that's not necessary. Where are the Kiev Jews who were shot in Babi Yar buried?"

"I don't know. Isn't your mother from Kiev? You should know where your relatives are buried."

"I don't know anything about them. I'll ask my mother."

Later on:

"Mom, do you know anyone from our acquaintances who was shot in Babi Yar?"

"The Germans shot Ukrainians in Babi Yar, but it was forbidden to talk about it in the Soviet Union, just like with the Holodomor, or else you would be arrested."

I am a descendant of Jews who lived in Kiev during World War II. But I know nothing about the Jews who were shot in Babi Yar. My relatives or acquaintances do not know either. My search led me to the grave of Ukrainians who were shot in Babi Yar and buried near the radio mast. This grave is visible in aerial reconnaissance photographs.

If you know who was shot and where they are buried, please tell me. I would be grateful. We can erect a monument at this location.

Re:We need to tell people ChatGPT will lie to them, not deba... by docmars on Saturday April 08, 2023 @10:34AM (#35493545)
Maybe with this knowledge, we can teach and encourage people to think more critically as they choose to adopt more of these tools?

I know more academic/intellectual types who are less willing to, than I do the average joe who seeks answers from all directions and discerns accordingly.

Re:We need to tell people ChatGPT will lie to them, not deba... by pinkcan on Saturday April 08, 2023 @01:16AM (#35490759)
ChatGPT scales better than "demagogues", so it does have more than a "little practical difference".
Re:We need to tell people ChatGPT will lie to them, not deba... by oezi on Saturday April 08, 2023 @01:35AM (#35490834)
> The part that's concerning about ChatGPT is that a computer program that is "confidently wrong"

I think we are just seeing Dunning-Krüger in the machine: It isn't smart enough to know it doesn't know. It likely isn't very far though.

Re:We need to tell people ChatGPT will lie to them, not deba... by Joeri on Saturday April 08, 2023 @02:18AM (#35490994)
Your characterization of “dumb people” as somehow being more prone to misinformation is inaccurate and disrespectful. Highly intelligent people are as prone to irrational thinking, and some research suggests even more prone. Go look at some of the most awful personalities on TV or in history, often they are quite intelligent. If you want to school yourself on just how dumb smart people are I suggest going through the back catalog of the “you are not so smart” podcast.

Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm.

ChatGPT is extremely poorly understood. People see it as a text completion engine but with the size of the model and the depth it has it is more accurate in my understanding to see it as a pattern combination and completion engine. The fascinating part is that the human brain is exclusively about patterns, combining and completing them, and those patterns are transferred between generations through language (sight or hearing not required). GPT acquires its patterns in a similar way. A GPT approach may therefore in theory be able to capture all the patterns a human mind can. And maybe not, but I get the impression nobody knows. Yet plenty of smart people have no problem making confident statements either way, which ties back to the beginning of this comment and ironically is exactly what GPT is accused of.

Is GPT4 at its ceiling of capability, or is it a path to AGI? I don’t know, and I believe nobody can know. After all, nobody truly understands how these models do what they do, not really. The precautionary principle therefore should apply and we should be wary of training these models further.

Re:We need to tell people ChatGPT will lie to them, not deba... by 23547234573457 on Saturday April 08, 2023 @02:21AM (#35491006)
This post is positive because HN tilts progressive rather severely. The reality is that "dumb people" are prone to a higher amount of irrational thinking because they lack awareness about a wider variety of topics. The point about respect is largely a tangent.
Re:We need to tell people ChatGPT will lie to them, not deba... by gersh on Saturday April 08, 2023 @02:20AM (#35491003)
It looks like it'll be competition for phony experts and politicians. I think it is easier to improve AI algorithms than deal human with human liars.
Re:We need to tell people ChatGPT will lie to them, not deba... by furyofantares on Saturday April 08, 2023 @02:39AM (#35491105)
> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence

Is this performance art?

I mean it could end up right but I think you basically just made it up and then stated it confidently.

Re:We need to tell people ChatGPT will lie to them, not deba... by shjake on Saturday April 08, 2023 @03:00AM (#35491191)
They didn’t state it particularly confidently though
Re:We need to tell people ChatGPT will lie to them, not deba... by furyofantares on Saturday April 08, 2023 @01:56PM (#35495881)
That's true, or at least not in the way ChatGPT does.

"Based on my understanding" does convey some lack of confidence in a sense. But it's also implying they might have more understanding than other posters here, when the content of their post indicates they have less.

Re:We need to tell people ChatGPT will lie to them, not deba... by nathias on Saturday April 08, 2023 @03:28AM (#35491334)
> unless

I don't think there is a possible unless, they are told repeatedly not to trust politicians, yet here we are ...

Re:We need to tell people ChatGPT will lie to them, not deba... by whiddershins on Saturday April 08, 2023 @03:33AM (#35491361)
The real problem is that smart people are even easier to program than stupid people and they do even more damage once they are programmed.
Re:We need to tell people ChatGPT will lie to them, not deba... by ThunderSizzle on Saturday April 08, 2023 @07:11AM (#35492240)
Many of the smarter people are still wrong on what happened on many topics in 2020. They were fooled by various arguments that flew in the face of reality and logic because fear and authority was used instead.

The people that avoid this programming isn't based on smart or stupid. It's based on how disagreeable and conscientious you are. A more agreeable and conscientious person can be swayed more easily by confidence and emotional appeals.

Re:We need to tell people ChatGPT will lie to them, not deba... by topologie on Friday April 14, 2023 @11:46AM (#35570853)
While reading your comment I realized that being "confidently wrong" is actually really human, so... yay?

I mean, that's one step closer to machines thinking like humans, right?

:)