This describes pretty accurately a non-trivial amount of people I have worked with.
- there’s no cited facts
- it boldly proclaims its conclusion
- which you’d only be able to verify as an expert
…so I’m having trouble understanding what you’re complaining about with chatGPT, when that seems to be the standard for discourse.
The comment I was replying to was “the errors you’re talking about are probabilistic if you read the literature” my response is “no they aren’t I have read the literature.”
Note that I’m talking about a specific class of error and proving a negative is difficult enough that I’m not diving through papers to find citations for something n levels deep in a hacker news thread.
Toolformer: Language Models Can Teach Themselves to Use Tools: https://arxiv.org/abs/2302.04761
PAL: Program-aided Language Models: https://arxiv.org/abs/2211.10435
TALM: Tool Augmented Language Models: https://arxiv.org/abs/2205.12255
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models: https://arxiv.org/abs/2201.11903
Survey of Hallucination in Natural Language Generation: https://arxiv.org/abs/2202.03629
Survey of Hallucination in Natural Language Generation only provided promising methods for detecting hallucinations in summarization tasks, which are of course much easier to detect. Searching arXiv for a list of non-reviewed papers that sound like they might be related to the topic at hand is fun debate strategy. But no one else is reading this far into an old thread, so I'm not sure who you're trying to convince.
None of these paper prove your claims about hallucinations, and most aren't even trying to. However, even if the errors that I'm saying aren't meaningfully probabilistic aren't hallucinations.
This does not mean it is a problem that is impossible to fix.
In the end, why do people believe what they believe? The answer is it connects with what they already believe. That's it. If you had a diet of what we call science, you'll have a foothold in a whole bunch of arenas where you can feel yourself forward, going from one little truth to another. If you are a blank slate with a bit of quackery seeded onto it, you end up believing the stars predict your life and that you can communicate with dead people with a Ouija board.
CGPT doesn't have an "already believe". It just has a "humans on the panel give me reward" mechanism, and all it's doing is reflecting what it got rewarded for. Sometimes that's the scientific truth, sometimes it's crap. All the time it's confident, because that's what's rewarded.
Have most people invested time in making sure their web search results are accurate?
It is dangerous for sure, but that's what you got.
That applies to my boss, college professors, Wikipedia, or my neighbor after a couple of beers. It's not designed to give the correct medical steps to diagnosis a brain injury.
I've heard people say they wouldn't use ChatGPT even if there was "only a 1 in a billion chance that it made their bank account details public"...
May I introduce you to some very real probabilities:
If you are living in America there is a 1 in 615 chance that your cause of death will be the result of an automobile accident.
So yes, it is unlikely that we will ever create a tool that can answer with 100% confidence. It is also unlikely that a manufacturing process will result in a widget that conforms to allowed tolerances 100% of the time.
However, in manufacturing this is understood. A defect of 3.4 per million widgets is considered an incredibly high process capability.
These tools are being made more reliable everyday. Please have a realistic goal in mind.
The problem is that an LLM can only be as reliable as its training material.
For example I wrote a blog post on the 2 generals problem. In the comments there is more text from inexperienced people asserting that a solution exists than there is text from my original article.
An LLM trained on that article and comments will always be wrong.
LLMs can be trained on smaller vetted training sets sure, but they also currently require massive amounts of data and there’s no guarantee that just waiting a few months or years will improve reliability enough to fix these issues.
There are a number of techniques you could try to improve your results.
But I get the distinct feeling that you are not interested in improving your results as you have a motivation to "prove" something about LLMs.
This becomes a problem when people start to treat LLMs as authoritative on facts.
So, it's sort of like driving a car based not on a complete map of the area, but a partial map where the missing pieces are filled in from statistical averages of things that appear generally in the world-- but not necessarily in that location. We all may be impressed at how close the predictions come to reality, but the only truly reliable thing is that those predictions, measured with precision, will be wrong. And if you rely on them without verifying them yourself, you might have a serious problem.
The “true” way to answer a proposition is not even remotely as simple as you’ve described.
I suspect I'm better at using the tool than most people. So if most people are not good at using the tool and then submitting their poor results to Stack Overflow, I can see the problem.
Acquiring and vetting information is a task of crucial importance situated in between journalism and science. Just trawling the internet and autocompleting the acquired driftwood is a weird approach to begin with. Doing so without transparency in private companies is simply folly.
In my experience with ChatGPT, it gets less wrong and more right than most humans, and it's able to find relationships among facts and concepts that are impossible with Wikipedia and Google.
It's extremely useful and powerful in its current form, in spite of its many limitations.
With other tools, we don't demand perfection to find them useful.
Just because a hammer sometimes misses the nail and puts a hole in the wall, just because Wikipedia sometimes gets vandalized with lies, doesn't mean hammers and Wikipedia aren't essential tools.
However, it provides an excellent opportunity for us mere humans to reflect upon the way we organize our society in regard to vital information processing. We are awfully bad at it. So bad actually, we can't even tell how it should be in the first place and are disturbed already by some piece of software telling nonsense.
They're turning the freakin' frogs gay!!!
And incidentally, it's the consequences of those errors that teach us not to repeat them... which is the same feedback loop we use to train AI in the first place!
That's pretty much the conclusion I've already come to. I have to verify everything ChatGPT tells me, so using it is pointless if I already know where/how to look something up.
We are just going to have ever more persuasive unreliable sources of information.
If you ask it question you have no way to know whether it’s right or not unless you already know the answer.
I’ve asked it many questions about my area of expertise, distributed systems, and it was often flat out wrong, but to a non expert it sounded perfectly plausible. I asked my my wife, a physician, to try it out and she reported the exact same problem. Some of the answers it gave had even veered off from just annoying to actually dangerous.
That doesn’t mean it can’t be useful, but using it to answer important questions or using it to teach yourself something you don’t already know is dangerous.