alecst
Maybe it’s a reason to start slowing down?
Understand that there’s more than one way to prove something, that there’s an art to making something understandable, and that humans tend to take the most understandable paths (to humans.)
Models often don’t do that. But in solving, or claiming to solve, important problems, they nevertheless remove a lot of the excitement and incentive towards working towards a solution humans would naturally find. Plus, a lot of these problems really don’t have any critical applications at the moment. They’re done for the love of the game.
Does that bother you? That your tax dollars pay people who do math out of curiosity?
Today it's a useful tool. But in not too long--a few years, perhaps--this tool will be able to outsmart you. If you think this is definitely not going to happen, then okay--we see the progress in LLMs differently. I see machines which, in a matter of three years, went from being able to do basic proofs, to math Olympiad problems, to century-old Millennium problems. I see no reason why progress will slow down.
Any "tool" which is that much smarter than you is no tool, any more than you are the "tool" of your pet dog.
I understand wanting to feel better and be optimistic about the future. I'm unfortunately not, and that makes a lot of people feel bad. Again, I'm sorry! I don't want to be a downer. But it's important to me that people take these risks seriously and don't minimize them with expressions like "we still have paper" and "it's a useful tool."
But since the richest and least trustworthy people on earth are on track to make an ultraintelligent thinking machine, the risks I'm worried about are: killing most jobs, concentrating most wealth, and putting ourselves at risk of literal extinction. All I could think when reading this was, "this person really did not think this all the way through." I'm sorry.
Would you bet your life, and everyone else's, to test that assumption?
Not to mention: you can scale up performance in simple ways: more data, more compute, and more weights. Those are the obvious ones, ignoring architectural changes.
And it minimizes the fact that these things are only getting smarter. Having an entity much smarter than you--could that entity seriously be considered your "tool"? Just your tool? Or are the people on the "just a tool" side of this debate not taking that possibility seriously?
In the 1960s there was recognition of some of the implications of
human intelligence. I. J. Good wrote [10]:
Let an ultraintelligent machine be defined as a machine
that can far surpass all the intellectual activities of any
any man however clever. Since the design of machines is one of
these intellectual activities, an ultraintelligent machine could
design even better machines; there would then unquestionably
be an "intelligence explosion," and the intelligence of man
would be left far behind. Thus the first ultraintelligent
machine is the _last_ invention that man need ever make,
provided that the machine is docile enough to tell us how to
keep it under control.
...
It is more probable than not that, within the twentieth century,
an ultraintelligent machine will be built and that it will be
the last invention that man need make.
Good has captured the essence of the runaway, but does not pursue
its most disturbing consequences. Any intelligent machine of the sort
he describes would not be humankind's "tool" -- any more than humans
are the tools of rabbits or robins or chimpanzees.
Throughout history and throughout cultures there's a universal: children, adults, and the elderly all like to feel like they have some kind of purpose in their community. Getting food, physical healing, emotional support, construction, passing on wisdom. All of these things are under threat.
I think you gravely underestimate how important this is, and how bored and dismal life would be if you didn't have some kind of purpose in your community.
Without the question, they might have never called. But because they called, we usually ended up just talking about other things too. Life. Girls. Work.
I rarely get calls like that anymore. It breaks my heart.
For the last few years, me and my friend have been working on a toy problem together. It's a problem that the most recent wave of LLMs can almost certainly solve and probably even one-shot. We now realize that no one's going to read our paper if we ever decide to publish our work. But we still use our weekly meetings to talk about things and make little progress here and there, come up with new questions, etc. It's bittersweet.
But what happens when a robot can do anything a human can? When a cheap machine can do anything you can do but faster and better, what makes you different from a horse in the 1920s?
This is what the article misses and what people have trouble accepting. Yea, this is a technological revolution, but unlike the Industrial Revolution, you can't just upskill your way out of it.
I skimmed the RAND paper too and didn't come away from this feeling good at all.
There's a middle-ground for AI doomage which doesn't involve killing every last human on earth. As AI rapidly becomes more capable it displaces jobs faster than people can be retrained.
I fear a global economic depression, wider wealth inequality than ever before, a loss of benefits due to much lower tax revenues and greater imbalance of power, dismal job prospects returning to non-automated labor--the list goes on really.
And to connect this to the article: the experts who claimed that AI capability would slow down because of some "scaling laws" or whatever--they've been wrong. If anything, the progress is faster than most of us imagined and is only accelerating.
ChatGPT came out 4 years ago. Look at what's possible today. Now imagine 4 years from today.