Useful for fiction, advertising copy, and literary criticism. Not so good for fact retrieval.
In other words, the temperature is controlling the variety of output, but of course doesn't affect what was fed into it in the first place. As the saying goes, Garbage-In, Garbage-Out .. even with a temperature of zero it's still going to be bullshitting since "predict next word" (language model) is fundamentally a bullshitting technology - just keep on spewing out words regardless of meaning.
I even asked it how the script could be improved and it made suggestions around adding error handling and making some hard coded names into command line parameters.
I asked it to give me code to implement the suggestions and it gave me working code.
It’s much better than you give it credit for.
OTOH I've also asked it what day of the week a given date was and received two different wrong answers depending on the exact phrasing of the question. I've also seen it confidently "explain" why taking 90% of a number and adding 10% of that back will get you to the original number...
The trouble is the output is a mix of truth and lies, and GPT has no way to distinguish between the two.
I once asked it write a Python script that lists all of the accounts in an AWS organization with a given tag key and value.
It confidently, initiated the SDK (boto3) and the correct object on the SDK (Organizations) and then it called a none existent function - “get_accounts_by_tag”.
The next day I asked it the same question and it got it right using a technique that I would have never thought of.
On the other hand, I asked it “given the following XML file and a DynamoDB table with the following fields, write a Python script that replaces the value node in the file where a corresponding key is found in the table with the value in the value field”.
The code was perfect.
When OpenAI had a way for training live data all big marketing companies would produce a ton of information just to get their „facts“ to ChatGPT.
But as a user you can’t compare different sources like you would do on Google and you only have this BS answer which is fancy but tells you to drink dish cleaner because studies have found out that dish cleaner makes stuff clean and clean is healthy.
Its lack of intelligence is not the problem. High intelligence doesn’t preclude misinterpretation, mis-remembering, or overestimating it’s own understanding.
If it was just acting as a search engine using english as the query language, then lack of intelligence wouldn't be an issue - the quality of output would just depend on the quality of the source as we're used to with search engines.
However, what ChatGPT is actually doing - due to it's fundamental nature as a language model (dealing only in word/language statistics) is effectively combining information from multiple sources, which of course is potentially very powerful if it knew HOW to utilize these variously sourced facts to construct a correct answer... but of course it doesn't, so it'll happily generate content mixed from factual and fantasy sources etc, or correct textbook programming exercises with buggy code from beginners it dredged up someplace. It's not just mixed sources though - it's the intelligence of how to take a bunch of raw facts and deduce something from them, and of course ChatGPT is not a deduction engine.
1) It's not a search engine, even if it behaves a bit like one. It's not "retrieving answers" to your questions (from sources that it could choose to cite). ChatGPT is really just a "language model", so it has no notion that what you're typing is even a question/query .. your input is just treated as sequence of words (which ChatGPT has zero understanding of), with ChatGPT's response then being a further sequence of words that it has calculated are (one) statistically probable continuation of what you typed (you can keep asking it for alternative answers, and it'll continue generating additional alternative statistically probable continuations).
The websites/etc that ChatGPT was trained on are just sources of language that it consumed in order to learn the statistics that let it make these continuation predictions. It's not memorizing "facts" from websites, just word statistics, and these are mixed in with the statistics from all the other sources it was trained on. If it generates the word "walk" as part of a response, it can't cite a source for that since there essentially is none - only a bazillion text sources it was trained on that collectively made the word "walk" a high probability continuation on the words it had generated leading up to that...
2) Even if ChatGPT had been designed to deal in "facts" (rather that words statistics) associated with specific sources, the bullshit problem isn't just knowing the varied reliability of the sources it was trained on, but how those "facts" are combined. To combine multiple facts and correctly deduce something new from them would require intelligence, but ChatGPT doesn't have any intelligence - it's just a statistical word generator, so the way it combines snippets from different sources is again just statistical word generation, with zero knowledge of the meaning of the words it is generating or whether it makes sense!
What makes ChatGPT seem semi-intelligent is that a lot of what it was trained on was text written by semi-intelligent humans, so the "sequence of words" it is generating, following the statistics of human speech, seems like something a human might say... until you start paying attention to the meaning of the words and realize it's often good-sounding garbage.