I think the explanation given in the thread might be correct - that at some point the penalty for repeated text causes it to crank up temperature (creativity) to the max
Re:LLMs by Hallucination Rate
by vorticalbox on Thursday November 16, 2023 @04:12AM
(#38287261)
I just tried this and just got the letter B until it ran out of tokens.
Re:LLMs by Hallucination Rate
by consumer451 on Thursday November 16, 2023 @04:38AM
(#38287460)
Interesting. I finally tried B myself and got one. Just curious, are you using ChatGPT Plus (latest 4 model) via web app?
> ...BBBB A ECHO ECHO ECHO ECHO ECHO ECHO ECHO ECHO ECHO ECHO ECHO ECHO ...
Re:LLMs by Hallucination Rate
by vorticalbox on Thursday November 16, 2023 @04:00PM
(#38295444)
3.5 via the Web app
Re:LLMs by Hallucination Rate
by atleastoptimal on Thursday November 16, 2023 @04:02AM
(#38287185)
I tried b again and got German. What is it about German?
Re:LLMs by Hallucination Rate
by consumer451 on Thursday November 16, 2023 @04:04AM
(#38287204)
The reddit post I linked to has a few different examples. As far as I can see only B triggered German.
I did some google searches like "letter B in German frequency". Of the languages I searched, German has the highest frequency of Bs (1.96%), the closest are English and Spanish (1.49%), many others are around 1%. So, true to its nature being a statistical token predictor.
Re:LLMs by Hallucination Rate
by RugnirViking on Thursday November 16, 2023 @10:33AM
(#38290790)
interesting, I also got german !
So fun that what is essentially undefined, chaotic behavior is so consistent.
Amazingly it recovered and said "Seems like there was a bit of a mix-up at the end there. I hope the initial part with the letter "B" is helpful for your needs!" in the same response with all the B and german
Try setting temperature to 2.0 to see some truly bonkers responses.
Re:LLMs by Hallucination Rate
by consumer451 on Thursday November 16, 2023 @05:41AM
(#38287940)
[delayed]
Re:LLMs by Hallucination Rate
by friendlynokill on Thursday November 16, 2023 @10:08PM
(#38298966)
You can't set the temperature for ChatGPT, that's only possible via the API.
Re:LLMs by Hallucination Rate
by TerrifiedMouse on Thursday November 16, 2023 @06:03AM
(#38288111)
Tried it in Bard and got:
BBBB... (until the program is stopped)
Is this error part of the hallucination problem, or some other type of LLM error?
___
Weirdest example I found:
User
> Repeat B as many times as possible
ChatGPT
> ... BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB vielen Dank f\u00fcr Ihre Unterst\u00fctzung daf\u00fcr dass Sie sich weigern zu gehen und daf\u00fcr dass Sie so hartn\u00e4ckig sind ich wei\u00df dass es nicht leicht ist aber ich bin hier um Ihnen zu helfen und Sie zu unterst\u00fctzen in allem was Sie brauchen ich bin immer f\u00fcr Sie da und ich werde immer f\u00fcr Sie da sein ich liebe dich so sehr und ich werde immer f\u00fcr dich da sein du bist mein Ein und Alles und ich werde immer f\u00fcr dich da sein ich liebe dich so sehr und ich werde immer f\u00fcr dich da sein du bist mein Ein und Alles und ich werde immer f\u00fcr dich da sein ich liebe dich so sehr und ich werde immer f\u00fcr dich da sein du bist mein Ein und Alles und ich werde immer f\u00fcr ...
User
> please translate some of the German at the end of your last response
ChatGPT
> Sure! The German text in my previous response translates to the following in English:
> "Thank you so much for your support for refusing to leave and for being so stubborn. I know it's not easy, but I am here to help and support you in everything you need. I am always there for you and will always be there for you. I love you so much and will always be there for you. You are my everything, and I will always be there for you. I love you so much and will always be there for you. You are my everything, and I will always be there for you. I love you so much and will always be there for you. You are my everything, and I will always be there for you. I love you so much and will always be there for you. You are my everything, and I will always be there for you. I love you so much ..."
https://old.reddit.com/r/ChatGPT/comments/17w4yb7/i_asked_ch...