But everyone doesn’t understand that, so there is a problem.
> I have yet to see anyone who treats its output as 100% fact
Yet they exist.
https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-s...
Not every human being on Earth knows what ChatGPT is, so that’s a non-starter.
> or everyone as in most reasonable people
That qualifier is doing some heavy lifting. It’s so vague it can be used to excuse anything¹. Not only reasonable people (however we’re going to define that) have access to the tool.
> ChatGPT tells you it can lie before you can even ask it anything.
And what you learn fast by doing user support of any tech tool is that most users don’t read instructions even if it’s shoved in their faces. I personally know people that think ChatGPT is the source of truth, despite that disclaimer.
In the same way that with a calculator, if you put bullshit in you will get bullshit out. In many cases you need to INPUT facts. This is automated with things like RAG.
I think this is why ChatGPT is extending deeply into RAG, because it matches more closely how users are expecting the tool to work.
Mimicking reasoning abilities is not the same thing as actually having/developing them.
That’s like saying “Oh, you didn’t get the right answer to that math question, you just mimicked giving me the right answer”. Yeah, but if I gave you the right answer, then I gave you the right answer, and I must’ve gotten it somehow.
I feel like we should have all moved on from the fact that LLMs don't have a very precise memory? This is (not entirely, but mostly) fixed by including the information you want to query in the prompt itself, like Bing does. ChatGPT is much better at summarization than precise fact recall.
And most of the time, it doesn't matter. When I'm using ChatGPT to learn or troubleshoot something, it's a jumping off point, not the end of the story. As long as everyone understands that, there's no problem. I have yet to see anyone who treats its output as 100% fact, and no AI company claims it either.