> string words together with some randomness consistent with an existing body of work

To best of my awareness (and observation of my inner monologue as well as what I say during a conversation), this is also very simplified but very true description of my thought processes. Obviously, my observations are very limited (and surely nothing new under the sun, just me not being aware), but in retrospective (and sometimes that happens even while my working memory is still pretty much there as I've barely said another word) I can spot the points at which my thought process was influenced by some unknown (non-observable) factor, and I've picked a different word or switched a line of thought, making it kind of obvious to me that I was stitching words in a line. Best noticed when thinking in a comfortable but foreign language. And I can also spew bullshit to myself, when I have limited knowledge of some subject or limited time to think something through but I'm pressed to blurt out something.

Obviously, there's nothing new. But I'm hoping all this AI hype would also inspire more research into Natural Intelligence research, even though those are very different fields (although, who knows, maybe efforts made into analyzing AI outputs/behaviors would reveal something about us, too).

It can be fun to use for creative writing -- why not? Creative writing is about making shit up.

But for gathering information it's the opposite of what we want. Integrating it with a search engine is not just ill-advised, it's incredibly stupid.

I tried to get it to write _1984_ erotic fanfiction, which it did - but everything it spewed out was a cliche. I imagine it had read all of FFN and AO3.

> gathering information

It will probably be like the Google infoboxes and “People Also Search” entries we’ve had for a while, only more superficially coherent.

It seems like the “fake it until you make it” craze has fully taken over the AI/ML scene.

If the outputs are boring it ain’t chatgpt at fault…

It is assessing the quality of the output as if a human had made it. Most of human writing is cliche. Expecting it to produce good output is, as you say, a matter of prompt engineering.

But I completely agree with you: it’s often spitting out boring stuff. And yes, it makes almost everything up based on language statistics and similarities.

Perhaps, we are just in a hype cycle? There are lots of specialty tools and bots available based on GPT-3 customized for different tasks. Complaining that ChatGPT or Bing+ aren’t equally suited for every task gets tiresome. Especially if more than 2/3 of the examples simply follow the pattern „junk in - junk out“.

It's not completely wrong all the time. Also the conditioning puts it into state where it has to respond with answers even if it doesn't know something. So coupling it with the tools for finding out accurate information is a logical next step. If it works out and you get highly plausible and true information most of the time, that's magical. Just because it's integrated doesn't mean you have to use it but many will. It'll keep getting better because that's how AI model have been doing at every benchmark, so even if you think it's not there yet, trends imply it might get there soon.

But it makes up plausible answers and even fake references. Which makes it worse than something which is always dead wrong.

And that doesn't seem to be addressable as long as the model is just stringing together the next most likely words as it learned them from some huge dataset. The answer "i don't know that" will always be rare, because nobody sits down and writes a blog post about how he could not find the solution to a problem.

Large language models are a true breakthrough and they are, in my view, much more useful than ChatGPT. They allow for some fundamental NLP tasks that can then be used as building blocks for truly intelligent systems.

Symbolic AI failed because it could not deal with complexity and fuzziness. Statistical learning can deal with complexity and fuzziness, but it is very limited when it comes to rigorous reasoning.

I believe that true AI will combine both ideas, and I have a strong hunch that neurosymbolic approaches will be crucial, but nobody really knows yet how to create a true neurosymbolic approach that seamlessly combines the strengths of symbolic and statistical approaches. My fear is that we through the baby out with the bath water when people are finally fed up with ChatGPT hype, and then we will have to wait a couple decades before the next serious attempt at AI is made.

The cynic in me would rather have the backlash (and a rude awakening for the industry), than the looming hyper-normalization of convincing-sounding bullshit/plagiarism/etc (and new, more convoluted forms of “saying the magic words to make the algorithm behave”) in the near future.

(Relatedly: Say what you will about the rationalist community, they appear to have really thought about a lot of this.)

But the point here is that as the subtlety and abstraction level of the material increases, the value of the statistical brute-forcing tapers off.

But one is less likely to apply such a tool to more challenging texts, in any case.

For me it is just another tool, that I can use to help me in certain areas. Not more. Not less.

If I was Sam Altman I would sell the thing asap before the hype dies out like with crypto.

I think he just did sell it to Microsoft for an additional $10bn.

This makes me think that in the future programming will be just about collecting data and then doing things with the data. We will all be data generators and collectors with a few math PHDs in the middle working on the activation functions and so forth. The rest of us will be stringing models together Stable Diffusion style or writing the billing code.

https://dallasinnovates.com/exclusive-qa-john-carmacks-diffe...

Is there any info on what these papers are?