Higher probably responses are not necessarily “truthful” or “logical” in any given context.

Here* you can see an example of what happens when you always choose the highest probability.

[*] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

It can halluciate at low temp too. e.g. mice don't necessarily eat cheese; it's just a common stereotype of what mice eat.

High temp is more likely to lead it into ranting on an unrelated topic. You ask it a question on what the mouse eats. It answers the question and then starts saying what lizards eat, or goes on a rant about food chains.

My favorite prompt to try so far to play with this effect is:

  Picture yourself in a boat on a river With tangerine trees and marmalade skies

It might veer off into a scenario where you're on a romantic boat trip or talk about how it is its favorite Beatles song and how it remembers them going on Ed Sullivan. :)

The temp affects the "color" of the next word which sends it on a tangent just like a person -- especially somebody with ADHD or playing an improve game as opposed to lip reading, for example.

No. I'd say everything the model does is a hallucination.

Some writers tend to split ChatGPT oputput into "correct answers" and "Hallucinated answers". If this is what you are thinking when saying "Is this part of why ChatGPT hallucinates?", then no. The model never knows if what it is saying is correct. It has a probability about how likely the words are to follow eachother, and that is about it.

The temperature changes how these probabilities are calculated, making otherwise less probable words more probable or vice versa.

However, the model doesn't choose the most likely word every time. Instead it rolls a dice to see where it wants to go next. If Cheese is 80% and Cookie is 20% it won't necessarily choose Cheese. It rolls a die, and maybe the outcome lands on the less likely Cookie.

The roll of the dice happen _after_ temperature has affected the calculation of probabilities.

Maybe it is this roll you are thinking of as hallucination?

You say it never knows if what it is saying is correct, but in my experience it tells you when it does and doesn't know about a concept. For example in one thread I had it give me some random words from linear algebra, then in another thread I combined these into the nonsense phrase "decomposed orthogonal eigenvector division" and asked it if it was familiar with this, and if so to define it. It said it wasn't familiar with it and asked for more information. I'd say that shows it knows what it doesn't know.

While it can be customary to trivialise this, the reality is that no "intelligence" is sure if what it's saying may be correct or constitute any kind of "knowledge". At this level a human can't claim to arbiter fact versus fiction.

Fundamentally, no one can be sure what they experience is not some kind of simulation, or a dreamlike state beneath some more real "reality". As such, assigning hierarchy to such a foundation lacking in tangible proof is arbitrary.

The point being, unless it's possible to conclusively and infallibly prove you are not hallucinating, you can't lay claim to having knowledge an AI is hallucinating, or not.

> Fundamentally, no one can be sure what they experience is not some kind of simulation, or a dreamlike state beneath some more real "reality".

To which I can only reply that I am not sure what I just read ...

> the reality is that no "intelligence" is sure if what it's saying may be correct or constitute any kind of "knowledge"

This is not true. An intelligence should almost by definition be able to recognize patterns, i.e. when things are similar or are the same. This is why we know that 1+1=2, since 1+1 and 2 are just different names for the same thing.

ChatGPT 'hallucinations' are a lack of this exact sort of consistency that makes it obvious that words are merely symbols and not representative of an actual thing. And how could it even know otherwise? ChatGPT is never exposed to things.

I'm certain in my knowledge of 1+1=2, but even if I wasn't, I know that 1+1=2 aren't just some ASCII symbols; but we're not entirely sure ChatGPT can be either of these.

> a words are merely symbols and not representative of an actual thing. And how could it even know otherwise? ChatGPT is never exposed to things.

An interesting argument, what does it mean for the disabled?

Eg, if a human had a fully functioning brain, but from birth could not sense anything except text sent to their brain, which they could only respond to with text, would this mean they're equivalent to a language model?

> but we're not entirely sure ChatGPT can be either of these.

Here you seem to agree with my point. We cannot be sure. Why?

> would this mean they're equivalent to a language model?

This is just a bad question. For one, we know that ChatGPT doesn't use neurological mechanisms. It has no proteins. It's not analogous. You're comparing meters to radians. I'm going to say no, a human without sensory organs is not just an LLM, not because of some complicated argument about how evolution imposes structure on the brain prior to any experiences it has, but because the premise is just bad.

> Here you seem to agree with my point. We cannot be sure. Why?

This is not the same type of 'sure' as in "we're not sure if there's life out there." It's 'sure' as in "we're not sure monkeys can build a bridge on their own."

Side note: when did it become standard rhetoric to say "you're agreeing with me now." ? Do you think I can't read? Or keep track of who I'm agreeing with? Or maybe you need to remind yourself?

It's made me somewhat interested in having an informative argument to actively disinterested in anything other than telling you how wrong you are. Can we keep it to reddit please?

How can we confidently state we know ChatGPT doesn't use neurological mechanisms, when we don't yet completely understand our own neurological mechanisms, nor emergent mechanisms that may be at work within ChatGPT?

At the same time, we can already comprehensively model some biological neurological systems in their entirety, in silicon. This will continue and the human mind will not be immune to such progress, making the question important.

I see it as a question that cuts to the core of the matter - long-standing, unanswered philosophical questions:

What does it mean to "perceive"? Can a machine be conscious? Are our biological processes meaningfully differentiable from equivalents when mimmicked by a machine? Are our "perceptions" special? Or not?

Putting the point another way:

We can't be objective about "perception" from within the constraints of perception.

While we routinely "imagine" an objective reality that exists beyond our perception, that we believe our perceptions reveal, there is no way to conclusively prove that's actually happening. Possibly, ever.

(As for the side note: I genuinely interpreted it that way, hence the word "seem", I didn't think about whether or not you'd missed it. I too assumed a conversation precisely unlike reddit... my default assumption here.)

You can't "cut to the core of deep philosophical questions" when you don't know the basics. For example, this

> we can already comprehensively model some biological neurological systems in their entirety, in silicon.

is just patently wrong.

> How can we confidently state we know ChatGPT doesn't use neurological mechanisms, when we don't yet completely understand our own neurological mechanisms, nor emergent mechanisms that may be at work within ChatGPT?

How can we confidently state that the sun isn't alive when we don't ourselves know what life is? I mean, it's got different layers like a cell. It has a source of energy, a lifecycle and it eventually dies. The sun has been around for much longer than chatGPT so I think we should answer this question first.

> While we routinely "imagine" an objective reality that exists beyond our perception

I'll just say this - if you really want to believe you're a brain floating in space, why don't you live life like it's GTA? Because obviously none of it is real, right?

Look up 'solipsism', this isn't a new or for that matter important philosophical position. Unless you want to write about how much fun whatever video game you're playing is in lieu of reality.

Solipsism implies there is only the self. However, it is possible that we all have similar enough brains that have evolved for us to share a false reality together, like Donald Hoffman's recent theories. On that note I do often wonder about the sentience of stars, photons, everything in between. So chatgpt is not much of a stretch and is exciting in that context.

The thing is that words are supposed to be useful. If a word can't tell the difference between photons, stars, and everything in between, then it's not very useful anymore. Calling all of these things 'alive' would just defeat the meaning of the word - we can't resuscitate a rock, we can't talk to photons, feed an atom - all these things that we associate with being alive are just rendered inapplicable.

> is just patently wrong.

Not really, here's one I found in seconds, that I hadn't even seen yet - where they specifically remark on the similarities to AI networks:

https://www.science.org/doi/10.1126/science.add9330

> How can we confidently state that the sun isn't alive when we don't ourselves know what life is? I mean, it's got different layers like a cell. It has a source of energy, a lifecycle and it eventually dies.

Well, I've learned from your previous comment you're not agreeing with me, but here's another instance I'd have thought you might be.

These are exactly the kinds of questions we should pose and explore. Are such a massive, complex, long-lasting sequences of nuclear processes "alive" or potentially intelligent? Might they have a detectable consciousness?

What are we but a vast sequence of chemical and electrical processes? In that sense, not much different.

(Science has indeed so far rather arbitrarily defined "alive", and this shows in some of the more controversial categories of "life", such as viruses.)

> I'll just say this - if you really want to believe you're a brain floating in space, why don't you live life like it's GTA?

I've never stated that, I've posed a relevant, important philosophical question I think lies at the centre of the issue of whether or not an AI can be considered to "hallucinate". As was the topic. Your response seems to be "no", and that's fine.

> I've never stated that, I've posed a relevant, important philosophical question

If you think this then go publish a paper and see how the community receives it. They decide what's important and relevant, not you.

Your side note is unwarranted - his statement about agreement was perfectly normal.

Again, do people need reminding of who agrees with who? What is the purpose other than to give points to each other? And I obviously wasn't agreeing with them, under any reading of the reply.

> The point being, unless it's possible to conclusively and infallibly prove you are not hallucinating, you can't lay claim to having knowledge an AI is hallucinating, or not.

This is very much true for more irrational (or less deterministic) "thoughts" in certain languages. However, if I asked you to write JavaScript or Python to "prove" your thoughts on the answer, it would be a simple matter of running the code you output on a state machine to determine the truths of the query.

  super-frog-of-adventure~> !calc 1+1
  athletic-giraffe|> 2
  super-frog-of-adventure~> !calc one plus one and output the answer
  athletic-giraffe|> Writing code.
  // filename: synth_calc_plus.js
  // version: 0.1.1
  // description: simple addition calculation
  var num1 = 1;
  var num2 = 1;
  var answer = num1 + num2;
  Sidekick(answer + " (units)", ", ");
  athletic-giraffe|> 2 (units)
  super-frog-of-adventure~>

That line of argument swiftly devolves into the case for solipsism.

Generally, you may preface any Internet discussion with: first, assume there an independent reality that the author and the readers share. Also causation and the forward arrow of time is true. (It occurs to me that this sounds like prompt engineering.)

In other words, you can never conclusively determine you’re not hallucinating (c.f. Hume), but let’s pretend we’re not for the sake of discussion.

Unfortunately, it also swiftly devolves into the issue being discussed...

It's not possible to prove whether or not a model can hallucinate or even think, based upon the premise that we have arbitrarily defined what constitutes "thought" (and so also "hallucination").

Can you prove an AI can or can't "think" in a way close enough to us that it could be considered to have basic ("human") rights? What if we built it out of biological material? And would that make any difference? Why?

This leads to: how may we objectively prove that the exclusivity of biological "consciousness", as we've defined it, is correct?

How may we prove either we or an AI can think, hallucinate, or "be conscious", when we are unable to objectively define the terms in a provable way?

I think it may be impossible, and as I said - I don't think it's a trivial concern. We may soon be faced with an entity that demands we respect it, and we need to have worked through as much of that discussion as possible, prior.

If you’re a behaviorist, you don’t care if it thinks or the internal machinery at all.

I think it hallucinates in scenarios where it either doesn't "remember" the answer or there is no answer.

For example: since the text generation is sequential, based on context the highest probability continuation in a situation might be:

"A famous paper that discussed this phenomenon is"

However, when the network gets to this point it doesn't know the name of such a paper. Maybe it already knew that it doesn't know the paper's name but the above sentence was generated despite that, since everything else was lower probability or it got sampled due to higher temperature. In any case, it can't go back and delete the sentence so it must make up something. So it ends up going with the most plausible sounding (highest sample probability) continuation, which ends up being a "hallucination".

Basically hallucinations are produced when the network "writes itself into a corner" due to the sequential nature of text generation.

definetly not.

Mice in nature don't eat cheese. So even the most favoured response is a hallucination.

In nature they also don't eat cookies then.

They favour grains over cheese. Cookies and baguettes (even for non-French mice) work better than cheese in mousetraps.

Peanut butter is widely recommended as the best mousetrap bait. I have had success with peanut butter.

I think hallucination is akin to encountering a problem on a test to which you don’t fully remember the right concept to solve it. You might attempt the problem with whatever little knowledge you can recall, but the answer is ultimately incorrect. Likewise, these LLMs exhibit such behavior when they hallucinate. I think what distinguishes the LLM’s hallucinations from humans is the confidence level. A human who is unsure of their answer might tell you that they’re not certain or they don’t know. LLMs like chatgpt, however, will just confidently make stuff up.