Which means that an LLM can probably technically come up with novel ideas, since novel ideas aren't some special category of things, it's just that LLMs are not very good at it.

It's such a banal suggestion it makes me think there could be a tension between the requirement to be creative, and the requirement that the next token be among the most probable.

"Hey make me a cool like marketing thing or whatever" isn't going to work.

Well, the rest mass notion in special relativity is just a natural derivation. It appears once you have the "real" ideas in place. And those aren't really "existing math" at all. It was a pure physics idea at root: "the universe's laws don't change if you are in motion" (or alternate framings like "you can't tell if you're moving from inside a moving box").

Well, it turns out that if you try to construct such a theory, you end up needing some different (pre-existing) math to help describe it. But the idea isn't math at all.

Then the question is "Can LLMs propose new well-framed, evocative theories like relativity", and the answer is sort of open. In point of fact human beings, to first approximation, can't do this either!

> Mass–energy equivalence states that all objects having mass, or massive objects, have a corresponding intrinsic energy, even when they are stationary. In the rest frame of an object, where by definition it is motionless and so has no momentum, the mass and energy are equal or they differ only by a constant factor, the speed of light squared (c²).

Now go check the "History" section of that article; which, by the way, takes up about half of it.

I'm trying to come up with other examples, but as the parent said, it then depends how pedantic we want to be e.g. take Poincaré's spacetime, but he'd still be working from two pre-existing ideas, "space" and "time"; the idea of combining the two is (feels?) quite novel and unexpected.

Going back some more, the notion of "space" (or that of "time") feels more primitive, less explainable in terms of other notions.