Epicycles gave a better description of the movement of the planets, based on the observations available, which was entirely falsifiable. They were eventually superseded, and that's science at work.
Newtonian mechanics has a model, beyond curve fitting.
BTW: I made this analogy between LLM and epicycles as a joke, but it's looking strangely isomorphic...
It's funny, because epicycles used to be the poster-child for ad hoc, overcomplex models with no conceptual basis... which is an exact match for LLM... but I don't recall the analogy being made. Not even in https://norvig.com/chomsky.html
We now discount that model because it's based on an incorrect geocentric model of the solar system, but that doesn't mean that the model wasn't a model...
[edit] The CHomsky link is interesting - I just listened to an interview where he pooh-poohs LLMs at great length.
This point: "Statistical models have been proven incapable of learning language; therefore language must be innate, so why are these statistical modelers wasting their time on the wrong enterprise?" is interesting in that context - in the recent interview, Chomsky is now unhappy that LLMs can learn /any/ language, even unnatural ones, and therefore aren't good tools for understanding human language. Quite the reversal. I personally think it's a 'science progresses one funeral at a time' kind of situation...
The first time this necessarily circular model was abandoned was with Kepler's laws of planetary motion, which correctly identified the motion of the planets as elliptical, that for the first time explained their apparent retrograde motion without the need for epicycles. Then Newton's theory of universal gravitation explained how the planets could possibly be moving on elliptical orbits. In fact, I believe Newton's theory of universal gravitation explained how the planets could be turning around the sun without crashing down, despite not having anything to hold them up. I think this was the first big mystery that the ancients tried to answer- hence the name of "firmament" for the universe.
And Newton's theory was not the end of the story of course.
A problem with epicycles is they are harder to falsify. If the model doesn't match new observations, just adjust it, or add another epicycle. In contrast, Newtonian gravity can hardly be tweaked at all. So when Mercury's orbit was slightly off, they knew something was wrong.
I'm not quite clear on how I feel about this. Geocentricity is a theory, in broad terms. It seems disrespectful to say adding epicycles makes it "not a theory". As you say, there was a theory that the planets actually moved in epicyclic motion. It wasn't just calculation to them.
(I want to stress that the idea of epicycles, the mechanical craftsmanship, and actual prediction of the planets are all amazing genius.)
Yet, having more parameters than data means the model doesn't explain in simpler terms, only restates. In this sense, it's "not a theory" (by Occam's razor). It seems enough epicycles can model anything: (3Blue1Brown Fourier Series) https://youtube.com/watch?v=bL0LV0Huj1s OTOH the epicycles did predict planetary motion, so they did capture some regularity... not sure what to think.
RE Chomsky: You can see it's like epicycles: with enough parameters, an LLM is like a numerical method for curve fitting, that doesn't explain the data (any more than a fourier transform does). Curiously, they do seem to predict very accurately... yet also generalize strangely ("hallucinate").
But it seems it's got to help! Even if only as a device, like a telescope. Also, from this interview with Terry Sejnowski (https://youtube.com/watch?v=XKC-4Tosdd8) there's instances where a technique was developed for a problem with Neural Nets, and an equivalent was found in the brain.
He also gives a Chomsky-like view: if you duplicated the human brain and it worked perfectly, you wouldn't have done any science if you didn't understand anything.
Well, that's the fundamental problem of modelling: that for any set of observations there's an arbitrary number of models that fit the data with great accuracy and even predict future observations well; and we don't know which one is the best in the long term.
The answer is that we should prefer not predictive models, but explanatory theories, that not only predict future observations but also explain why those observations should be expected to be made.
For example, the epicyclical model did not explain anything: it said nothing about why the planets should move on circular orbits with epicycles. Kepler's laws didn't explain anything because they didn't say why the planets should move on ellpitical orbits. Newton's law of universal gravitation explained it all in one stroke: because gravity. And that's why we consider Newton the greatest scientist of his era, not Kepler, not Coppernicus, not Gallileo, but Newton, because he explained the world and didn't just describe it.
>> BTW Chomsky's point E (which I'd never heard of), the last and most minor, was based on Gold's work.
Gold's negative learnability result was a huge upheaval that led directly to the current paradigm of machine learning. Chomsky used it to argue about his universal grammar but linguistics was only one of the two fields that Gold's result turned upside down.
And it was a negative result. As I say in another comment, science gives you the tools to know when you're wrong and that's how progress is made, when we find out where we were wrong before.
With epicycles, it took almost two thousand years before we figured out where the model was wrong. Let's hope that it doesn't take that long with LLMs and neural nets also, because I doubt we have another couple thousand years to spare on a wild goose chase.
Do you know where Chomsky refers (directly or indirectly) to Gold? I've been searching for a reference for some time.
Just to make sure I didn't hallucinate all that, I had an admittedly perfunctory search online and I could find this paper:
https://proceedings.neurips.cc/paper/2002/file/04ad5632029cb...
Whose introduction describes how Gold's result is considered to support the arguments for linguistic nativism from the poverty of the stimulus. Then again, the author doesn't seem to be a linguist himself and he doesn't give any more specific references, so I'm now a little worried; and your question remains un-answered.
Have you tried wading through Chomsky's early work on linguistics? I don't have the courage to. The closest I've got to is I have a friend who has read a couple of Chomsky's linguistics books. My friend is making a living as an astrologist now so maybe that's a bit of a warning there :P
"Enlightened" was my bad choice of a word. I get overexcited when I think of how much we have learned in the past couple thousand years and I forget that we mainly learned how little we know. Or can explain!
I'd like to think explaining means giving a model simpler than the observations. But this can be true of a predictive model, that offers no "why". Another commenter pointed out that epicycles do simplify - so they do "explain" in this sense.
Maybe an advantage of an explanatory theory is in revealing more of the "black box", giving more ways to check the theory. (But I'm not sure how this could apply to Newton's gravity, since the only observations were outcomes. And no plausible way to "experiment".)
> If it wasn't for the ancients stumbling and fumbling in the dark for millennia
Is there any evidence that the epicyclic models helped scientific understanding, even indirectly? Later theories didn't seem to build on it. I wonder if it actually detoured understanding, with its misleadingly impressive accuracy, so that understanding would have progressed more quickly without it.
Thinking of pg's "great work" (https://news.ycombinator.com/item?id=36550615): to be the Newton of neural nets would seem the most ambitious aspiration of our times. But it took a bunch of geniuses just to get to Newton... and it seems an even harder problem than planetary motion. Though a difference is neural nets are based on actual neurons (loosely!).
It's looking like working human-level AI will precede understanding... perhaps by those 2000 years?
Do you know where Chomsky says this exactly? I've been looking for how/if his argument is the same as Gold's.
And the heliocentric model still has plenty of unexplained parameters: the major and minor radii for each body. (Not to mention those pesky perturbations in mercury's orbit.)
> Chomsky is now unhappy that LLMs can learn /any/ language, even unnatural ones, and therefore aren't good tools for understanding human language
That's also not what he has said: He said they aren't useful for understanding the human language faculty, in other words, understanding how people are able to have language. As he says it obviously can't be the same way as LLMs because LLMs are able to learn languages humans can't learn.
PS: I would be happy to connect, you can find my socials in my bio.
Because science gives you the tools to know that you're wrong. If you're a good scientist, you will be wrong _all the time_. That's how science advances: one mistake at a time. But you can't make mistakes if all you ever do is doing stuff with computers, like beating all the benchmarks, because that is a meaningless result judged by its own, self-chosen, measure of success that can never fail; and so can never inform.
Peer review is also not science, but it has been great to catch errors in my papers. Not in conferences, mind. I try to stay away from conferences. Everybody flocks to conferences because of quick turnaround, instant gratification. Journals have the good reviewers who can take their time understanding your work and helping you find where you've gone wrong. "Reject with encouragement to resubmit" is the best review result I ever got.
>> I judge things as "what you can do", not "what can you predict".
The goal of science is not to make predictions, but to understand how the world works, and why. Put that into instrumentalism's pipe and smoke it.
Moreover, in social contexts, religion also "works" in many senses. A person asked me what me what can he do about depression and feeling of meaninglessness. Science would prescribe anti-depressants, medicate him and he would both have side-effects and the problem would never be cured. I suggested that if he were religious, the meaninglessness would immediately go away and he would need no medication, no external interventions.
So again, judging religion by "what can it do", it can successfully give people meanings. Which is quite non-trivial thing that science fails to do to many people. So religion can be presumed to be acting on some truths regarding what drives and gives people meanings. Religion successfully demonstrated an understanding of that, you don't need statistical analysis or peer review to see that, it's plainly obvious. That knowledge just doesn't carry on to other stuff like how celestial bodies move, and there's no reason to think it does.
I love how you're the spokesperson for "science".
Ironically, when you look at how you describe the merits of peer review, you sound quite like an instrumentalist yourself.
Also quite ironically, the new science religion keep on pushing this funny narrative that science is about "being wrong all the time" which is like the complete opposite of scientific history. All of the established science is just the successes. Look at the great physicists. There is no mistake in Newton's laws. Or in Einstein's. They are just as valid today as they have been back then, in the areas where they were demonstrated and studied.
How did we reach this apologetic science? "good scientist will be wrong all the time"? That sounds like gaslighting scientists in a failed system rather than accurate historical statement. Science advances because someone eventually succeeds. In the grand scheme of things, there are no mistakes. There are corrections, there are expanding the domain of applicability. But the mistakes are sent to the trashbin of history, and if a failed system decided to gaslight you into rationalizing failure, stop listening to it and start seeking success.
A model being incomplete is just another way of saying it's wrong.
The models they came up with are wrong, but still useful.
The hard sciences would like a word.
I judge things as "what you can do", not "what can you predict". The only demonstration of knowledge and understanding is being able to do something. Not predict. Not "scientific method" and ridiculous "peer review" (actually peer pressure), not blind trials and not rigorous statistical analysis.
In the end of the day, you either manage to do something or you don't. So much of so called science had lost all contact with reality because our judgement of success isn't successfully doing something, it is successfully jumping through "scientific" hoops. Look at string theory and social sciences. The scientific process, instead of being a tool, became the purpose. It became a stamp of validity to seek. A stamp of validity with gatekeepers in the academia, in the peer review process, in the media coverage afterwards all the way to social media censorship and "fact checkers".
What used to be the frontier of creative people became a stagnant beurocratic machine worshipped like a new religion. The side of the heretics burned at the stake became the ones crying out heresy.
Enjoy your new brand of science. I'll stick to the older brand of heretics and mad men which did whatever it was the prevailing orthodoxy told them to avoid doing and thinking, and I'll remind you that the only real reason those are remembered is because they did something useful, not because of the social traditions they adhered to or the rigorous scientific standards they followed.