They literally generate stuff that's averages of other stuff.
It's not interesting in the least
Usually commentators say that they have heard so and so version of the joke, or that it's a repeat, but they did not
Both from opus. Sol/astra could not be funny. The versions that worked I had fed top posts from r/jokes as references and gave abstract directions.
However, the core premises were not new even if the construction was. And it's a lot of work to get something useful out.
I'm curious, did you test with the 200 jokes if it could guess which were the beat ones according to a human audience?
In images, music, games, it usually can't. So the process of validation is offloaded to the audience. I'm very interested in seeing where this takes different platforms. I bet Spotify can pull off such a validation process. Music is passive, tolerating a bad song doesn't require much from you. And they have a massive audience. They can try out AI generated mess with a very low success percentage without saturating their playlists and alienating their users. But can Steam tolerate it? Trying out a game is a much bigger effort than listening to a song picked for you. Customers are already reacting to the number of demos in Steam Next Fest. The saturation point is much lower.
And what about jokes? What happens to r/jokes if even 10 standup comedians are validating 200 jokes per day?
And we see it in software too. QA is already being offloaded to the customer.
I want to see where this goes, even if I don't like it.
But if you find a media where AI can also validate the output instead of having to offload it, that's going to be very valuable!
In these LLM comparisons, people tend to compare them to the best humans, not average or below average. It’s a kind of motivated reasoning, with the goal of deprecating the abilities of LLMs.
But I think humor is pretty easy to rate for a human. Did it make you laugh? Yes? Then it might be a good one. This simple human filter radically improves your efficiency as a joke validator, leaving any LLM far behind.
For pictures I'm not sure if humans are much better. For music, yes, after a few listens (recognizing a hit usually takes time). For games, definitely yes.
Interesting because pure creativity is the most prominent thing they can't do by design.
An LLM is like a low IQ person with vast knowledge.
If they were really stupid, say like a Prolog program or a solver, only stating the goal was enough,
https://www.anthropic.com/research/riemann-zeta
My point is that it does matter who's driving the AI but it also matters the route. Just like F1 champion Max Verstappen is a ridiculously better driver than me at driving an F1 car around a track, if we're just driving a friend to the airport, it doesn't really matter if it's Max or me driving, so long as we don't crash.
People can follow what they did. The documents OpenAi produced in their big drop are hard to read because they are poorly written in weird ways, not because they are beyond human intelligence. They didn't brute force in the normal sense. AI doesn't get tired, so it can try 1000 promising ideas, but it's like 1000 Ph.D.-level mathematicians trying different ideas, not 1000 calculators trying all combinations.
The biggest results -- quasi-Riemann, no Siegel zeros, and Unique Games Conjecture -- have already been digested by experts. No Siegel zeros in particular already has like five different write-ups, and Tao has written a exposition on his blog.
It gels because they solved about 700 problems out of 4000 attempts. And they threw a ton of resources at each. So "per problem" it's less than 1 in 5, "per token" it's probably not a great record. I mean it's astonishing that it's possible at all of course. But on average, even an expert with a top model will probably have an underwhelming response to the vast majority of prompts.
Ie. the LLM itself isn't as "smart" as the combination of LLM, Lean, looping, search and vast gpu/cpu-time.
While they're not great at explaining their own proofs, some models are good at explaining older, human-written proofs.
How many people on this planet are experts on every topic?
>An LLM is like a low IQ person with vast knowledge.
Are you aware that frontier models have scored in the 135-195 IQ range when they have been tested? How smart are you?
If you know how they work and how they are trained this makes perfect sense.
Going out and searching the web gets around model cutoffs, so fixes the latency of knowledge issue a bit, but they’re still finding statistical modes of what they’ve seen before.
“Stochastic parrot” is seen as a crude and dismissive jibe at LLMs, but it’s actually a concise observation of the reality.
E.g. consider the best answer describes how to achieve something with some CLI, but the CLI's changed and the command is now broken. It can discover that, combine with the 'best answer' about what changed in the CLI, and fix it to work for the original problem even if that's completely novel and there were no extant answers about solving the problem with the new CLI form, or that it had broken that use case.
It doesn't change fundamentally how they work of course, but 'thinking', creating that feedback loop, made them significantly more useful IMO.
It’s going to narrow in on a statistically better answer but it’s not going to create novel insight beyond the arguments made from compressed snapshots of existing knowledge.
There is no “thinking”. Just more attention, which is apparently, all you need.
> it’s not going to create novel insight beyond the arguments made from compressed snapshots of existing knowledge
So all of the maths breakthroughs were not novel?
What I’ve found LLMs to be OK at when it comes to novel insight is being given a goal to work towards (“prove or disprove this”), with access to a large body of sources and it can then chase down 1000s of options no human would bother with.
This is why security and maths proofs seem to be an area where it can accelerate things. However I know from my own work in these areas it can still hallucinate and so the output needs careful checking.
Is that novel ideation? Or is it self-organising grid search? It’s more efficient than getting 1000 mathematicians to do the work, but it’s not doing something “new”, it’s building on existing work.
And yes, I know what “attention” means, I was making a pun. And it still stands up: they’re not in some deep thought about this and reflecting on new ideas, they’re just hammering away at everything in the literature.
And it still might be bullshit.
To demonstrate:
"Are you saying that a" https://news.ycombinator.com/item?id=49642931
"complex stateful system" https://news.ycombinator.com/item?id=14881439
"cannot generate a" https://news.ycombinator.com/item?id=45956162
"state that is not already" https://news.ycombinator.com/item?id=43013637
"recorded in text" https://news.ycombinator.com/item?id=44731577
"somewhere?" https://news.ycombinator.com/item?id=50391679
A novel state, obtainable by combining six existing ones.
ignoring the fundamentally dishonest representation of LLMs in this subthread..
if humanity was given a tool that could restructure already existing and known knowledge and facts into novel results with a statistically better answer automatically it wouldn't be as a minor a win as you make it sound; it would be tremendous.
Hard disagree.
Claude is able to come up with solutions in very niche areas where there is little discussion on the web, because it takes steps to digest the source of (for instance) frameworks and libraries, and can then use/advise on lots of esoteric options, edge-cases and all sorts of stuff that doesn't get discussed on the open web.
It’s great at bread and butter stuff, but not at specialised ideation.
If you know differently, please share examples as maybe we can learn something here.
This is fundamentally different from your earlier, stronger claim that - "The very best LLM can answer is the answer you can expect from an online forum like Reddit or a book/paper published more than a year ago."
I use a less popular java framework quite a lot, Micronaut (I want to move away from it, but that's a different discussion).
When trying to solve problems with its behaviour, I can watch as Claude digests the framework source to come up with suggestions, identify bugs that are causing behaviour regressions, and generally unlock stuff that I've previously found very frustrating. Frustrating because the documentation covers a tiny amount of what's in there, and frustrating because there's little online discourse about any of it to refer to.
Is this earth shattering in itself? Probably not. But it's certainly better than you can find from "an online forum like Reddit or a book/paper published more than a year ago".
It’s kinda like the project where someone generated a lot of midi files for all possible pop riffs to copyright them. If you have a start and endpoint, rules for navigation, and unlimited computes, you can brute force anything. The nice stuff is when you made exploration efficient.
That is not remotely how the vast majority of the proofs have worked. You don't know what you're talking about, but that seems par the course for these discussions.
Especially as the pure gruntwork involved in acquiring the knowledge required to merely understand the details of the mathematics involved represent at least a significant fraction of a human lifetime.
And unless this human process of knowledge acquisition continues, eventually no one will be left to understand and attach any significance to the results machines produce.
Which is what they claim to have done.
The LLMs are vastly superior for any subject matter with even the slightest hint of danger because while both LLMs and Reddit will whine at you and talk down to you the LLM will actually knock it off when told.
Such a good turn of phrase.
> Briefly stated, the Gell-Mann Amnesia effect works as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward-reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them. In any case, you read with exasperation or amusement the multiple errors in a story-and then turn the page to national or international affairs, and read with renewed interest as if the rest of the newspaper was somehow more accurate about far-off Palestine than it was about the story you just read. You turn the page, and forget what you know. —Michael Crichton
Edit: sibling comment explains it much more plainly.
I now have a newspaper that doesn't give me that feeling.
Many comments here are speaking in absolutes and being myopic about what's "novel". The truth is yes, LLMs output is essentially derivative, but it's also true that a lot of what we consider novel is also derivative. The two aren't mutually exclusive at all.
Had a novel idea? You didn't get it from thin air, everyone is standing on the shoulders of everyone who came before. LLMs most of all, but that doesn't mean it can't bring disparate bits of word-encoded knowledge together and burp out something we'd consider "novel" if a person had made the same connections.
That was actually the basis of some of the mathematics proofs. It made connections from existing material. Nothing a human couldn't do looking at the same material, but we're talking speed and scale here - same as all computational processes.
A calculator does math faster but we don't call it sentient or feel inferior or dismiss the results. The only difference is LLMs are intentionally nondeterministic, as that behaviour can be usefully harnessed, pun intended.
So there's no need to argue about what's novel or not. LLMs can make novel connections via raw speed and breadth of disparate data. The human brain is more complex and "imaginative" for whatever your definition of that is - I mean fundamentally one either believes we are just physical neurons, or that there's a spiritual aspect on top, the end result is the same.
A six sided dice can also be novel if it happens to unexpectedly roll your phone number. It's a novel occurrence, same as an idea coming to mind that, TO YOU, seems novel... but we can't ask the other 8 billion people on the planet so who are we to judge what's a novel idea anyway?
Actually, we do. We do dismiss calculator results. We are all generally aware of a calculator's abilities. We know it has limits. So when a calculator outputs 9999e999999 we know to distrust that result. It was something out of the calculators concept of understanding. So too a variable like Pi. "3.14159265359" is not Pi and every user knows it is an approximation not be used for anything too important or precise. The same should apply to LLMs... but their limitations remain nebulous and ever-changing.
As a concrete example, when you ask Claude Sonnet 5.5 to generate a small-to-medium-sized one-shot Python project, it will always fall into the same code patterns. Same structures of functions. Same file layout. Same package layout. They're not even good patterns, because if you throw a few common linters at it (e.g. flake8, isort, mypy) there are a ton of linting errors to clean up.
The only way that it escapes this mediocrity is through context. If you give it enough guardrails or sample data (e.g. Python code to emulate), it will start to go off in a "novel" direction that is defined by that context.
Now, what's the most likely way that "novelty" will be generated in an automated system like this? Randomly pulling in context and going for a random walk through the compressed language-based knowledge storage. It remains to be seen whether that's an effective way of producing more than just parlor tricks for investors. Human+LLM is overwhelmingly more novel and effective than only LLM.
If that's the only limitation that you're putting forward than LLMs have already won over humans. If you're not hoping for some quantum level limitations then it's already over for us.
[LLMs can’t jump]: https://openreview.net/forum?id=klU4737opt
What's your point?
The syntax of math is overloaded so you need less syntax to write out something that would be verbose in English ...no one wants to make sense of arithmetic expressed in German syntax. May as well write all software in brainfuck.
Written literacy on a core language like we focus on today is not the historical norm. Creoles and local dialects have vanished in the US as we hyper-normalize everyone into Children of the Corn like carbon copy zombies of character traits
Ultimately any claim that there exist only combinations of old ideas must tackle this chicken-and-egg problem convincingly. For example- explain art. There was no art until it appeared fully formed and out of nowhere in early anatomically modern humans' dwellings (Neanderthals also had art but it was simpler and it is subject to the same chicken-and-eggness). So where did art come from? What pre-existing idea was combined to bring it into existence from non-existence?
Btw, since I've had this conversation before- if the answer is that we looked around us in the world and saw things; yes, but we didn't see art. Where did art come from? Where did the idea come from of abstracting shapes and colours so that they resemble the shapes and colours of existing things, without actually being those things, or being composed of pieces of those things? How about numbers? How about writing? How about all our technology and science? Where did those things come from? What did we combine to create them?
Obviously I think that the argument that there are only new combinations is wrong: we are a creative species. We can create new things. Somehow. It would be very interesting to understand how. We have not yet created machines that can match us in that and we can't understand the "how" by studying the machines we have now.
So it's not there are zero new ideas, but so few compared to the whole of the intellectual domain, that we can say 'there are no new ideas under the sun' and be basically correct to within a small rounding error. And AI is sufficiently ingenious to serve the needs of the vast majority of humanity, even though we'll still need to find/cultivate/inspire some new genius humans if we want to have more truly new ideas in the future.
Its fairly plausible that the original ideas could be sensory gradient-triggered impulses in simpler creatures' proto-minds that (physical and cultural) evolution has gradually built up through more complex neural machinery supporting more complex representational language.
> where did art come from
Similarly, its not a stretch to see art as built from earlier attempts at representation in physical space - starting with mundane attempts at information exchange by colony insects or pack creatures. The breadth and depth of expression has evolved alongside more complex neural machinery and greater variety in media through technology. There's a hidden lineage in creatures' expressive capabilities that may not be evident when we look at a Caravaggio today, sure.
There's nothing terribly chicken-and-egg about these issues (though the resolution is the same - "chicken" and "egg" co-evolved through iterative selection pressures, one didn't precede the other).
All that said, I concur that its a fantastic and mysterious topic to unpack, but it doesn't follow for me that humans are anointed as uniquely "creative" or that there's necessarily something metaphysical at play.
Oh no, I don't think you understand how big the problem space actually is.
https://en.wikipedia.org/wiki/Kruskal%27s_tree_theorem
A very simple set of 3 possible combinations blows up into a possibility that is 2 to the 2 teterations 1000. This number cannot be explained to the human mind realistically.
> What pre-existing ideas were combined to bring it into existence from non-existence?
This depends what you believe about existence. For example if you believe in mathematical existences then 'simple' exploration of the problem space can lead you to find anything that could possibly exist in the conditions our physical universe allows algorithmically. And the number of those things are for a practical matter, infinite.
>we are a creative species. We can create new things. Somehow.
You are viewing this from a very human perspective scale though. Your individual cells are massively creative containing solutions to a staggeringly huge number of problems, including ones they've not seen before (take the immune system for example), yet how 'intelligent or creative' would you consider your cells to be.
Simply put you're ignoring algorithmic creativity and problem space exploration as it is, until you at least look at how simple systems create infinite complexity already.
Again: what's the number of combinations of elements of ∅?
(∅ is the empty set, a.k.a. {})
Occasionally they do something that surprises you. If you put them inside a search loop and spend huge sums of money you'll get some nuggets of value, but for most hard problems the cost isn't worth it.
And they work best with a lot of steering.
From that POV they're exactly like human culture. Most scientific papers are trash, a few are gamechangers. Same with the arts, etc.
But we've gone from 'Can write sentences' to 'Can do frontier math problems' in maybe five years.
We absolutely do have AGI now. It's not very good AGI, but it's probably better than a lot of people were expecting.
Being better than humans at everything isn't AGI, it's ASI. A lot of the comments here seem to be lamenting that we don't have that yet, which is an interesting data point about how far the goal posts have moved.
it is nothing but combining known knowledge that begets new.
I think a harness for building hierarchical summaries with tooling for expanding and searching history is what's missing for this. We're getting there with Recursive Language Models and continuous compaction systems like the one NVidia just published:
The ingenious aspect seems to be that we've found simpler ways to apply computing power to a broader range of problems, which it's important to note is as much a testament to the developers of proof assistants like Lean as it is to LLM developers, to say nothing of the countless mathematicians over many centuries that contributed training data.
A lot of math theorems are discovered that way by humans btw: find a difficult problem in a domain, map it to another domain, apply existing theorems there, go back to the original domain. The only difference is an LLM would have to search for an interesting problem, but that can be done by brute-forcing search at the seams and validating the problem does not previously exists. And there is no new TO YOU for the LLM: it can get help from a vast amount of helper domains.
IMHO there's a shot at having category theory really unifying a large chunk of maths in the next decade. It'll be a big ugly ball of Lean, but a queryable one at that.
Declarations about what LLMs can't do "by their nature" historically have a terrible shelf life.
On the other hand, there's no evidence that work will dry up merely because it becomes more efficient, or even if it does, that this cannot lead to a better world. The trick is finding a way to prevent rapid changes in the nature of work from destroying too many lives, and that's a very human problem machines alone can't solve.
They know far more than any human ever could: they've read all the books. Their knowledge is enormous. And that often includes how to apply it in standard ways, and sometimes in slightly non-standard ways. But whenever something gets complex, detailed and nuanced, they can't keep up with a human. It becomes a shallow illusion that's easily punctured by just a bit of critical thought.
You mention system architecture, but even for writing code, the code it writes is often horrible. It makes it work because it has a feedback loop that rapidly tests it and ensures it will work, but the actual code it writes down is often not something I would feel comfortable showing to another programmer.