1) It's not a search engine, even if it behaves a bit like one. It's not "retrieving answers" to your questions (from sources that it could choose to cite). ChatGPT is really just a "language model", so it has no notion that what you're typing is even a question/query .. your input is just treated as sequence of words (which ChatGPT has zero understanding of), with ChatGPT's response then being a further sequence of words that it has calculated are (one) statistically probable continuation of what you typed (you can keep asking it for alternative answers, and it'll continue generating additional alternative statistically probable continuations).
The websites/etc that ChatGPT was trained on are just sources of language that it consumed in order to learn the statistics that let it make these continuation predictions. It's not memorizing "facts" from websites, just word statistics, and these are mixed in with the statistics from all the other sources it was trained on. If it generates the word "walk" as part of a response, it can't cite a source for that since there essentially is none - only a bazillion text sources it was trained on that collectively made the word "walk" a high probability continuation on the words it had generated leading up to that...
2) Even if ChatGPT had been designed to deal in "facts" (rather that words statistics) associated with specific sources, the bullshit problem isn't just knowing the varied reliability of the sources it was trained on, but how those "facts" are combined. To combine multiple facts and correctly deduce something new from them would require intelligence, but ChatGPT doesn't have any intelligence - it's just a statistical word generator, so the way it combines snippets from different sources is again just statistical word generation, with zero knowledge of the meaning of the words it is generating or whether it makes sense!
What makes ChatGPT seem semi-intelligent is that a lot of what it was trained on was text written by semi-intelligent humans, so the "sequence of words" it is generating, following the statistics of human speech, seems like something a human might say... until you start paying attention to the meaning of the words and realize it's often good-sounding garbage.
Useful for fiction, advertising copy, and literary criticism. Not so good for fact retrieval.
In other words, the temperature is controlling the variety of output, but of course doesn't affect what was fed into it in the first place. As the saying goes, Garbage-In, Garbage-Out .. even with a temperature of zero it's still going to be bullshitting since "predict next word" (language model) is fundamentally a bullshitting technology - just keep on spewing out words regardless of meaning.
I even asked it how the script could be improved and it made suggestions around adding error handling and making some hard coded names into command line parameters.
I asked it to give me code to implement the suggestions and it gave me working code.
It’s much better than you give it credit for.
OTOH I've also asked it what day of the week a given date was and received two different wrong answers depending on the exact phrasing of the question. I've also seen it confidently "explain" why taking 90% of a number and adding 10% of that back will get you to the original number...
The trouble is the output is a mix of truth and lies, and GPT has no way to distinguish between the two.
I once asked it write a Python script that lists all of the accounts in an AWS organization with a given tag key and value.
It confidently, initiated the SDK (boto3) and the correct object on the SDK (Organizations) and then it called a none existent function - “get_accounts_by_tag”.
The next day I asked it the same question and it got it right using a technique that I would have never thought of.
On the other hand, I asked it “given the following XML file and a DynamoDB table with the following fields, write a Python script that replaces the value node in the file where a corresponding key is found in the table with the value in the value field”.
The code was perfect.
When OpenAI had a way for training live data all big marketing companies would produce a ton of information just to get their „facts“ to ChatGPT.
But as a user you can’t compare different sources like you would do on Google and you only have this BS answer which is fancy but tells you to drink dish cleaner because studies have found out that dish cleaner makes stuff clean and clean is healthy.
Its lack of intelligence is not the problem. High intelligence doesn’t preclude misinterpretation, mis-remembering, or overestimating it’s own understanding.
If it was just acting as a search engine using english as the query language, then lack of intelligence wouldn't be an issue - the quality of output would just depend on the quality of the source as we're used to with search engines.
However, what ChatGPT is actually doing - due to it's fundamental nature as a language model (dealing only in word/language statistics) is effectively combining information from multiple sources, which of course is potentially very powerful if it knew HOW to utilize these variously sourced facts to construct a correct answer... but of course it doesn't, so it'll happily generate content mixed from factual and fantasy sources etc, or correct textbook programming exercises with buggy code from beginners it dredged up someplace. It's not just mixed sources though - it's the intelligence of how to take a bunch of raw facts and deduce something from them, and of course ChatGPT is not a deduction engine.
I made a proof of concept this weekend - https://github.com/VikParuchuri/researcher . There are some issues, but it's very useful.
It's a very clever proof of concept. Not exactly a large language model.
- Search using Google
- Run some filters to exclude SEO spam, etc.
- Scrape the pages that are returned
- Find chunks of text likely to align with the answer (comparing embeddings)
- Feed the most likely chunks into GPT-3 to get a summary
I want to experiment with a local model next, versus using GPT-3.
Why do you say this? It's going to be one of many similar products, even then, it's impressive, it's fun, but is it really useful yet? I think we have to wait and see?
By the way, I like ChatGPT and have a lot of fun with it.
It doesn't need to cite sources, because it has learned to make them up on the fly.
Liars care about the truth; they want to subvert it. Large language models like ChatGPT don't "lie", they produce bullshit that has zero connection to what is or what isn't, but just sounds like something someone would say in that context.
It's possible, even likely, that ChatGPT and the like will help people formulate queries; but if it answers them and people somehow trust its answers, we're all doomed.
There is no doubt that chatGPT is the future. It is certainly perfectible, but the existing basis is a revolution in progress.
In my opinion, there are two essential things missing for chatGPT to become the perfect replacement for Wikipedia and Google: - The ability to activate a "system 2" or slow thinking (theorized by Daniel Kahneman) - The ability to cite sources
And the cherry on the cake would be the ability to interact with images