News for hackers. Stuff that matters.
Re:Simply explained: How does GPT work? by brazzy on Thursday April 06, 2023 @02:46PM (#35472635)
Almost all people almost never have truly original ideas. When asked to "tell me an idea [you] have never heard before", they will remix stuff they have heard to get something that "feels" like it's new. In some cases they'll actually be wrong and reproduce something they heard and forgot about hearing, but remember the concept. Most of the time, the remix will be fairly superficial.

And remixing stuff it has heard before is exactly what ChatGPT is doing. What it sucks at is the "feels like it's new" part, but fundamentally it would be quite easily capable of creating output that combines concepts with maximally negative correlation, the only thing that's truly missing is the ability to interpret the prompt as an instruction to do that.

Re:Simply explained: How does GPT work? by pulvinar on Thursday April 06, 2023 @03:13PM (#35473025)
Certainly. I mean we've seen all 26 letters before-- ChatGPT is just remixing them.

How does one actually measure novelty, without having to know everything first?

Re:Simply explained: How does GPT work? by brazzy on Thursday April 06, 2023 @11:51PM (#35477846)
The entire strength of large language models like GPT is that they do know a frighteningly good approximation of everything, in terms of having been trained on text written about it.
Re:Simply explained: How does GPT work? by fl7305 on Thursday April 06, 2023 @04:22PM (#35473848)
> And remixing stuff it has heard before is exactly what ChatGPT is doing.

Check out my "the confetti has left the cannon" example above.

https://news.ycombinator.com/item?id=35346683

Maybe still "remixing", but it sure feels like new to me.