Actually, we do. We do dismiss calculator results. We are all generally aware of a calculator's abilities. We know it has limits. So when a calculator outputs 9999e999999 we know to distrust that result. It was something out of the calculators concept of understanding. So too a variable like Pi. "3.14159265359" is not Pi and every user knows it is an approximation not be used for anything too important or precise. The same should apply to LLMs... but their limitations remain nebulous and ever-changing.
As a concrete example, when you ask Claude Sonnet 5.5 to generate a small-to-medium-sized one-shot Python project, it will always fall into the same code patterns. Same structures of functions. Same file layout. Same package layout. They're not even good patterns, because if you throw a few common linters at it (e.g. flake8, isort, mypy) there are a ton of linting errors to clean up.
The only way that it escapes this mediocrity is through context. If you give it enough guardrails or sample data (e.g. Python code to emulate), it will start to go off in a "novel" direction that is defined by that context.
Now, what's the most likely way that "novelty" will be generated in an automated system like this? Randomly pulling in context and going for a random walk through the compressed language-based knowledge storage. It remains to be seen whether that's an effective way of producing more than just parlor tricks for investors. Human+LLM is overwhelmingly more novel and effective than only LLM.
If that's the only limitation that you're putting forward than LLMs have already won over humans. If you're not hoping for some quantum level limitations then it's already over for us.
[LLMs can’t jump]: https://openreview.net/forum?id=klU4737opt
What's your point?
The syntax of math is overloaded so you need less syntax to write out something that would be verbose in English ...no one wants to make sense of arithmetic expressed in German syntax. May as well write all software in brainfuck.
Written literacy on a core language like we focus on today is not the historical norm. Creoles and local dialects have vanished in the US as we hyper-normalize everyone into Children of the Corn like carbon copy zombies of character traits
Ultimately any claim that there exist only combinations of old ideas must tackle this chicken-and-egg problem convincingly. For example- explain art. There was no art until it appeared fully formed and out of nowhere in early anatomically modern humans' dwellings (Neanderthals also had art but it was simpler and it is subject to the same chicken-and-eggness). So where did art come from? What pre-existing idea was combined to bring it into existence from non-existence?
Btw, since I've had this conversation before- if the answer is that we looked around us in the world and saw things; yes, but we didn't see art. Where did art come from? Where did the idea come from of abstracting shapes and colours so that they resemble the shapes and colours of existing things, without actually being those things, or being composed of pieces of those things? How about numbers? How about writing? How about all our technology and science? Where did those things come from? What did we combine to create them?
Obviously I think that the argument that there are only new combinations is wrong: we are a creative species. We can create new things. Somehow. It would be very interesting to understand how. We have not yet created machines that can match us in that and we can't understand the "how" by studying the machines we have now.
So it's not there are zero new ideas, but so few compared to the whole of the intellectual domain, that we can say 'there are no new ideas under the sun' and be basically correct to within a small rounding error. And AI is sufficiently ingenious to serve the needs of the vast majority of humanity, even though we'll still need to find/cultivate/inspire some new genius humans if we want to have more truly new ideas in the future.
Its fairly plausible that the original ideas could be sensory gradient-triggered impulses in simpler creatures' proto-minds that (physical and cultural) evolution has gradually built up through more complex neural machinery supporting more complex representational language.
> where did art come from
Similarly, its not a stretch to see art as built from earlier attempts at representation in physical space - starting with mundane attempts at information exchange by colony insects or pack creatures. The breadth and depth of expression has evolved alongside more complex neural machinery and greater variety in media through technology. There's a hidden lineage in creatures' expressive capabilities that may not be evident when we look at a Caravaggio today, sure.
There's nothing terribly chicken-and-egg about these issues (though the resolution is the same - "chicken" and "egg" co-evolved through iterative selection pressures, one didn't precede the other).
All that said, I concur that its a fantastic and mysterious topic to unpack, but it doesn't follow for me that humans are anointed as uniquely "creative" or that there's necessarily something metaphysical at play.
Oh no, I don't think you understand how big the problem space actually is.
https://en.wikipedia.org/wiki/Kruskal%27s_tree_theorem
A very simple set of 3 possible combinations blows up into a possibility that is 2 to the 2 teterations 1000. This number cannot be explained to the human mind realistically.
> What pre-existing ideas were combined to bring it into existence from non-existence?
This depends what you believe about existence. For example if you believe in mathematical existences then 'simple' exploration of the problem space can lead you to find anything that could possibly exist in the conditions our physical universe allows algorithmically. And the number of those things are for a practical matter, infinite.
>we are a creative species. We can create new things. Somehow.
You are viewing this from a very human perspective scale though. Your individual cells are massively creative containing solutions to a staggeringly huge number of problems, including ones they've not seen before (take the immune system for example), yet how 'intelligent or creative' would you consider your cells to be.
Simply put you're ignoring algorithmic creativity and problem space exploration as it is, until you at least look at how simple systems create infinite complexity already.
Again: what's the number of combinations of elements of ∅?
(∅ is the empty set, a.k.a. {})
Many comments here are speaking in absolutes and being myopic about what's "novel". The truth is yes, LLMs output is essentially derivative, but it's also true that a lot of what we consider novel is also derivative. The two aren't mutually exclusive at all.
Had a novel idea? You didn't get it from thin air, everyone is standing on the shoulders of everyone who came before. LLMs most of all, but that doesn't mean it can't bring disparate bits of word-encoded knowledge together and burp out something we'd consider "novel" if a person had made the same connections.
That was actually the basis of some of the mathematics proofs. It made connections from existing material. Nothing a human couldn't do looking at the same material, but we're talking speed and scale here - same as all computational processes.
A calculator does math faster but we don't call it sentient or feel inferior or dismiss the results. The only difference is LLMs are intentionally nondeterministic, as that behaviour can be usefully harnessed, pun intended.
So there's no need to argue about what's novel or not. LLMs can make novel connections via raw speed and breadth of disparate data. The human brain is more complex and "imaginative" for whatever your definition of that is - I mean fundamentally one either believes we are just physical neurons, or that there's a spiritual aspect on top, the end result is the same.
A six sided dice can also be novel if it happens to unexpectedly roll your phone number. It's a novel occurrence, same as an idea coming to mind that, TO YOU, seems novel... but we can't ask the other 8 billion people on the planet so who are we to judge what's a novel idea anyway?