> Another dumb question: can the model be smarter than the data?

There's a mathematical sense in which it cannot be: the data-processing inequality (https://en.wikipedia.org/wiki/Data_processing_inequality). The extent to which that sense is totally counterintuitive to how things normally work in ML nowadays has led to investigating different foundations for information theory: https://arxiv.org/abs/2601.03220.

Data isn't "smart", it doesn't contain intelligence. I would argue that LLMs are functionally intelligent, they have learned to what functionally amounts to thinking by training on text (and images, in some cases). They are capable of performing experiments autonomously and make decisions based on out-of-distribution data. Besides (current) lack of online learning, they are limited by their biases and working memory - but so are humans, and I'd like to see a convincing argument on why these mathematical laws should not also apply to humans, which are arguably intelligent.

"Smarter" is ill-defined.

Are you smart if you see that cognitive scientists had something figured out, connected it to something in your field of chemistry and made a breakthrough treatment for some disease?

Does any of our existing intelligence build on completely unknown information? You could argue that any new solution/emergent intelligence is an association of existing information in new ways.

It also brings us to what is brute forcing? Humans are brute forcing stuff most of the time. You will have 10 stupid thoughts for every good thought. 10 stupid ideas for every good idea. It's very rare, if not near impossible, that a non-mathematician with no prior knowledge, goes into a room on day 1 of deciding to think about a conjecture they don't know anything about, and solve it on the blackboard. They need to use their collect associations, formalize their axioms, think of first principles (known information), and associate it.

So to answer your question you need to answer:

1) What does "smarter" refer to? Is it accumulation of information about the universe? Is it solving tasks useful for the human in the world? Something else? Is it acting as an agent with it's own goals and interests in the universe? What if the goals are hard for humans to understand?

2) What does "genuinely new approaches" refer to? Is it fine to use math axioms to prove things? Does interpolation between two existing solutions count as new? At what granularity of concept does novelty happen (Does using two higher abstraction concepts, like combining some new, modern neural network architecture with a database count as a new thing), even if the underlying lower abstraction of information as storage, latency and compute is not novel?