Well, to the extent that a human-level intelligence is an individual, anyway. We ourselves are probably a mixture-of-experts in some sense.
Also, for the purposes of talking about the phenomenon of recursive self-improvement, individual vs society isn't the end of analysis. Part of the reason AI recursive self-improvement is concerning is that people are worried about it happening on much faster than societal timescales, in ways that are not socially tractable like human societies are (e.g. if our society is "improving" in a way we don't like, we or other humans can intervene to prevent, alter, or mitigate it). It's also important to note that when we're talking about "recursive self-improvement" when it comes to AI, the "self" is not a single software artifact like Llama-70B. The "self" is AI in general, and the most common proposed mechanism is that an AI is better than us at designing and building AIs, and the resulting AI it makes us even better at designing and building AIs.
Though… still don't think it's true. Isn't "society is self improving" what they call Whig history?
I.e. "You can only use the memory which you currently use" would be a weird artificial constraint not relevant in practice.
Are you arguing that all AI models are using the same network structure?
This is only true in the most narrow sense, looking at models that are strictly improvements over previous generation models. It ignores the entire field of research that works by developing new models with new structures, or combining ideas from multiple previous works.
The exception is when you care about efficiency (in training or inference costs) but at the limit or if you care about "better" then you don't.
Training to predict internet dump can only give you so much.
There's a paper called something like "learning from textbooks" where they show that a small model trained on high-quality no-nonsense dataset can beat a much bigger model at a task like Python coding.
>Yes, OpenAI was literally founded by a computer worshipping religious cult.
What cult is this?
OpenAI founders: Ilya Sutskever, Greg Brockman, Trevor Blackwell, Vicki Cheung, Andrej Karpathy, Durk Kingma, Jessica Livingston, John Schulman, Pamela Vagata, Wojciech Zaremba, Sam Altman. All of them come from software tech industry and academic research circles, not evidence of interest in HPMOR or Yud.
They do.
Humans can learn from new information, but also by iteratively distilling existing information or continuously optimizing performance on an existing task.
Mathematics is a pure instance of this, in the sense that all the patterns for conjectures and proven theorems are available to any entity to explore, no connection to the world needed.
But any information being analyzed for underlying patterns, or task being optimized for better performance, creates a recursive learning driver.
Finally, any time two or more humans compete at anything, they drive each other to learn and perform better. Models can do that too.
Ok. So then I guess it isn't "just a belief that magic".
Instead, it is so true and possible that you think it is actually obvious!
I'm glad you got convinced in a singular post that recursive self improvement, in the obvious way, is so true and real that it is obviously true and not magic.
Better intelligence can be defined quite easily: something which is better at (1) modeling the world; (2) optimizing (i.e. solving problems).
But if that would be too general we can assume that general reasoning capability would be a good proxy for that. And "better at reasoning" is rather easy to define. Beyond general reasoning better AI might have access to wider range of specialized modeling tools, e.g. chemical, mechanical, biological modeling, etc.
> if it is true it'll be obvious in a way that doesn't make it sound interesting anymore.
Not sure what you mean. AI which is better at reasoning is definitely interesting, but also scary.
> they just come from training bigger models on the same data.
I don't think so. OpenAI refuses to tell us how they made GPT-4. I think a big part of it was preparing better, cleaner data sets. Google tells us that specifically improved Gemini's reasoning using specialized reasoning datasets. More specialized AI like AlphaGeometry use synthetic datasets.
> Yes, OpenAI was literally founded by a computer worshipping religious cult.
Practice is the sole criterion for testing the truth. If their beliefs led them to better practice then they are closer to truth than whatever shit you believe in. Also I see no evidence of OpenAI "worshipping" anything religion-like. Many people working there are just excited about possibilities.
> Humans don't have a "recursive self-improvement" ability.
Human recursive self-improvement is very slow because we cannot modify our brains' at will. Also spawning more humans takes time. And yet humans made huge amount of progress in the last 3000 years or so.
Imagine that instead of making a new adult human in 20 years you could make one in 1 minute with full control over neural structures, connections to external tools via neural links, precisely controlled knowledge & skills, etc.
You need to apply Wittgenstein here.
This appears to be true because you haven't defined "better". If you define it, it'll become obvious that this is either false or true, but if it is true it'll be obvious in a way that doesn't make it sound interesting anymore.
(For one thing our current "AI" don't come from "writing code", they just come from training bigger models on the same data. For another, making changes to code frequently makes it worse instead of better.)
> I guess people working there believe in magic.
Yes, OpenAI was literally founded by a computer worshipping religious cult.
> People believe that self-improvement might happen when AI is around human-level.
Humans don't have a "recursive self-improvement" ability.
Also not obvious that an AI that was both "aligned" and "capable of recursive self-improvement" would choose to do it; the new "improved AI" would not be them and so how do you know it's aligned anymore?