The problem is suddenly most of what ChatGPT can do is getting drowned out by "I asked for this incredibly easy Google search and got nonsense" because the general public is not willing to accept 80-90% on what they imagine to be very obvious searches.
The way things are going if there's even a 5% chance of asking it a simple factual question and getting a hallucination, all the oxygen in the room is going to go towards "I asked ChatGPT and easy question and it tried to gaslight me!"
-
It makes me pessimistic because the exact mechanism that makes it so bad at simple searches is what makes it powerful at other usecases, so one will generally suffer for the other.
I know there was recently a paper on getting LMs to use tools (for example, instead of trying to solve math using LM, the LM would recognize a formula and fetch a result from a calculator), maybe something like that will be the salvation here: Maybe the same way we currently get "I am a language model..." guardrails, they'll train ChatGPT on what are strictly factual requests and fall back to Google Insights style quoting of specific resources
Worse is really better, huh.
For me it's been useful for taking highly fragmented and hard-to-track-down documentation for libraries and synthesizing it into a coherent whole. It doesn't get everything right all the time even for this use case, but even the 80-90% it does get right is a massive time saver and probably surfaced bits of information I wouldn't have happened across otherwise.