Minus the urgency, scientific process, well-defined goals, target dates, public ownership, accountability...
Some of you do. The rest of us are left with the consequences.
Even the president needs someone else to push a button (and in those rooms there's also more than one person). There's literally no human that can do it alone without convincing at least 1 or 2 other people, depending on who it is.
https://amp.cnn.com/cnn/2017/11/18/politics/air-force-genera...
The urgency was faked and less true of the Manhattan Project than it is of AGI safety. There was no nuclear weapons race; once it became clear that Germany had no chance of building atomic bombs, several scientists left the MP in protest, saying it was unnecessary and dangerous. However, the race to develop AGI is very real, and we also have no way of knowing how close anyone is to reaching it.
Likewise, the target dates were pretty meaningless. There was no race, and the atomic bombs weren't necessary to end the war with Japan either. (It can't be said with certainty one way or the other, but there's pretty strong evidence that their existence was not the decisive factor in surrender.)
Public ownership and accountability are also pretty odd things to say! Congress didn't even know about the Manhattan Project. Even Truman didn't know for a long time. Sure, it was run by employees of the government and funded by the government, but it was a secret project with far less public input than any US-based private AI companies today.
> There was no nuclear weapons race; once it became clear that Germany had no chance of building atomic bombs, several scientists left the MP in protest
You are forgetting Japan in WWII and given casualty numbers from island hopping it was going to be a absolutely huge casualty count with US troops, probably something on the order of Englands losses during WW1. Which for them sent them on a downward trajectory due to essentially an entire generation dying or being extremely traumatized. If the US did not have Nagasaki and Hiroshima we would probably not have the space program and US technical prowess post WWII, so a totally different reality than where we are today.
Well, you didn't provide any evidence. Island hopping in the Pacific theater itself took thousands of lives, imagine what a headlong strike into a revanchist country of citizens determined to fight to the last man, woman and child would have looked like. We don't know how effective a hypothetical Soviet assault would have looked like as they had attacked sparsely populated Sakhalin only. What the atom bomb succeeded was in convincing Emperor Hirohito that continuing the war would be destructively pointless.
WW1 practically destroyed the British Empire for the most part. WW2 would have done the same for the US in your hypothetical scenario, but much worse.
https://en.wikipedia.org/wiki/Operation_Downfall#:~:text=Tru....
The big problem that McArthur and others pointed out is that all the Japanese forces on the Asian mainland and left behind in the Island Hopping campaign through the Pacific were unlikely to surrender unless Japan itself was definitively defeated with the central government capitulating and aiding in the demobilization.
From their perspective the options were to either invade Japan and force a capitulation, or go back and keep fighting it out with every island citadel and throughout China, Indochina, Formosa, Korea, and Manchuria.
I'd say they were equal. We were worried about Russia getting nuclear capability once we knew Germany was out of the race. Russia was at best our frenemy. The enemy of my enemy is my friend kind of thing.
It seems pretty irresponsible for AI boosters to say it’ll happen within 5 years then.
There’s a pretty important engineering distinction between the Manhattan Project and current research towards AGI. At the time of the Manhattan Project scientists already had a pretty good idea of how to build the weapon. The fundamental research had already been done. Most of the budget was actually just spent refining uranium. Of course there were details to figure out like the specific design of the detonator, but the mechanism of a runaway chain reaction was understood. This is much more concrete than building AGI.
For AGI nobody knows how to do it in detail. There are proposals for building trillion dollar clusters but we don’t have any theoretical basis for believing we’ll get AGI afterwards. The “scaling laws” people talk about are not actual laws but just empirical observations of trends in flawed metrics.
Agreed. Do they?
Demis Hassabis said 50/50 it happens in 5 years.
Jensen Huang said 5 years.
Elon Musk said 2 years.
Leopold Aschenbrenner said 5 years.
Matt Garman said 2 years for all programming jobs.
And I think most relevant to this article, since SSI says they won’t release a product until they have superintelligence, I think the fact that VCs are giving them money means they’ve been pretty optimistic in statements about about their timelines.
Still, you’d have to be quite an idiot to wait for the third time to listen eh?
Besides, the winners get to decide what’s a war crime or not.
And when the US started mass firebombing civilian Tokyo, it’s not like they were going to be able to just ‘meh, we’re good’ on that front. Compared to that hell, being nuked was humane.
And I don’t say that lightly.
As made quite apparent by, as you note, kamikaze tactics and more.
The Bomb was a cleaner, sharper, and faster Axe than invading the main island.
That it also sent a message to the rest of the world was a bonus. But do you think they would have not used it, if for example the USSR wasn’t waiting?
Of course not, they’d still have nuked the hell out of the Japanese.
What does AGI do? AGI is up against a philosophical barrier, not a technical one. We'll continue improving AI's ability to automate and assist human decisions, but how does it become something more? Something more "general"?
With transformers, demonstrated first by LLMs, I think we've shown that the narrow-general divide as a strict binary is the wrong way to think about AI. Instead, LLMs are obviously more general than any previous AI system, in that they can do math or play chess or write a poem, all using the same system. They aren't as good as our existing superhuman computer systems at these tasks (aside from language processing, which they are SOTA at), not even as good at humans, but they're obviously much better than chance. With training to use tools (like calculators and chess engines) you can easily make an AI system with an LLM component that's superhuman in those fields, but there are still things that LLMs cannot do as well as humans, even when using tools, so they are not fully general. One example is making tools for themselves to use - they can do a lot of parts of that work, but I haven't seen an example yet of an LLM actually making a tool for itself that it can then use to solve a problem it otherwise couldn't. This is a subproblem of the larger "LLMs don't have long term memory and long term planning abilities" problem - you can ask an LLM to use python to make a little tool for itself to do one specific task, but it's not yet capable of adding that tool to its general toolset to enhance its general capabilities going forward. It can't write a memoir, or a book that people want to read, because they suck at planning or refining from drafts, and they have limited creativity because they're typically a blank slate in terms of explicit memory before they're asked to write - they have a gargantuan of implicitly remembered things from training, which is where what creativity they do have comes from, but they don't yet have a way to accrue and benefit from experience.
A thought exercise I think is helpful for understanding what the "AGI" benchmark should mean is: can this AI system be a drop-in substitute for a remote worker? As in, any labour that can be accomplished by a remote worker can be performed by it, including learning on the job to do different or new tasks, and including "designing and building AI systems". Such a system would be extremely economically valuable, and I think it should meet the bar of "AGI".
But they can't, they still fail at arithmetic and still fail at counting syllables.
I think that LLMs are really impressive but they are the perfect example of a narrow intelligence.
I think they don't blur the lines between narrow and general, they just show a different dimension of narrowness.
You are incorrect. These services are free, you can go and try it out for yourself. LLMs are perfectly capable of simple arithmetic, better than many humans and worse than some. They can also play chess and write poetry, and I made zero claims at "counting syllables", but it seems perfectly capable of doing that too. See for yourself, this was my first attempt, no cherry picking: https://chatgpt.com/share/ea1ee11e-9926-4139-89f9-6496e3bdee...
I asked it a multiplication question so it used a calculator to correctly complete the task, I asked it to play chess and it did well, I asked it to write me a poem about it and it did that well too. It did everything I said it could, which is significantly more than a narrow AI system like a calculator, a chess engine, or an image recognition algorithm could do. The point is it can do reasonably at a broad range of tasks, even if it isn't superhuman (or even average human) at any given one of them.
>I think that LLMs are really impressive but they are the perfect example of a narrow intelligence.
This doesn't make any sense at all. You think an AI artifact that can write poetry, code, play chess, control a robot, recommend a clutch to go with your dress, compute sums etc is "the perfect example of a narrow intelligence." while a chess engine like Stockfish or an average calculator exists? There are AI models that specifically and only recognise faces, but the LLM multitool is "the perfect example of a narrow intelligence."? Come on.
>I think they don't blur the lines between narrow and general, they just show a different dimension of narrowness.
You haven't provided an example of what "dimension of narrowness" LLMs show. I don't think you can reasonably describe an LLM as narrow without redefining the word - just because something is not fully general doesn't mean that it's narrow.
how much is (0.2 + 0.1) * 10?
The result of (0.2+0.1)×10 is approximately 3, with a slight rounding difference leading to 3.0000000000000004.
For poetry: counting syllables is a significant part of most poetry forms, so if you can't count syllables, you can't do poetry.
Let's say you want a 5-7-5 haiku, this is ChatGPT
write a 5-7-5 haiku about windstorms
Fierce winds howl and spin,
Branches bend, clouds race the sky,
Storm leaves quiet calm.
LLMs are not general, but they show that a specific specialization ("guess next token") can solve a lot more problem that we thought it could.
>[AI system]s are not general, but they show that a specific specialization ("[process sequential computational operations]") can solve a lot more problem that we thought it could.
Or if you really want:
>Humans are not general, but they show that a specific specialization ("neuron fires when enough connected neurons fire into it") can solve a lot more problem that we thought it could.
This is just sophistry - the method by which some entity is achieving things doesn't matter, what matters is whether or not it achieves them. If it can achieve multiple tasks across multiple domains it's more general than a single-domain model.
Or he will simply shift goalposts, and call some LLM superintelligent.
Edit: actually I'm not sure if AIXItl is technically galactic or just terribly inefficient, but there's been trouble making it faster and more compact.
In any case anyone who is completely sure that we can/can’t achieve AGI is delusional.
The fact is many things we’ve tried to develop for decades still don’t exist. Nothing is guaranteed
Basically, unless you can show humans calculating a non-Turing computable function, the notion that intelligence requires a biological system is an absolutely extraordinary claim.
If you were to argue about conscience or subjective experience or something equally woolly, you might have a stronger point, and this does not at all suggest that current-architecture LLMs will necessarily achieve it.
1. There is a chemical-level nature to intelligence which prevents other elements like silicon from being used as a substrate for intelligence
2. There is a non material aspect to intelligence that cannot be replicated except by humans
To my knowledge, there is no scientific evidence that either are true and there is already a large body of evidence that implies that intelligence happens at a higher level of abstraction than the individual chemical reactions of synapses, ie. the neural network, which does not rely on the existence of any specific chemicals in the system except in as much as they perform certain functions that seemingly could be performed by other materials. If anything, this is more like speculating that there is a way to create energy from sunlight using plants as an existence proof of the possibility of doing so. More specifically, this is a bet that an existing physical phenomenon can be replicated using a different substrate.
What evidence can you provide to back up the statement of this "significant possibility"? Human brains use neural networks...
Modern ANN architectures are not actually capable of long-term learning in the same way animals are, even stodgy old dogs that don't learn new tricks. ANNs are not a plausible model for the brain, even if they emulate certain parts of the brain (the cerebellum, but not the cortex)
I will add that transformers are not capable of recursion, so it's impossible for them to realistically emulate a pigeon's brain. (you would need millions of layers that "unlink chains of thought" purely by exhaustion)
even if we bought this negative result as somehow “proving impossibility”, i’m not convinced plasticity is necessary for intelligence
huge respect for richard sutton though
More specifically: it is highly implausible that an AI system could learn to improve itself beyond human capability if it does not have long-term plasticity: how would it be able to reflect upon and extend its discoveries if it's not able to learn new things during its operation?
(That said, I agree plasticity is key to the most powerful systems. A human race with anterograde amnesia would have long ago gone extinct.)
If I'm a human tasked with editing video (which is the field my startup[0] is in) and a completely new video format comes in, I need the long term plasticity to learn how to use it so I can perform my work.
If a sufficiently intelligent version of our AI model is tasked with editing these videos, and a completely new video format comes in, it does not need to learn to handle it. Not if this model is smart enough to iterate a new model that can handle it.
The new skills and knowledge do not need to be encoded in "the self" when you are a bunch of bytes that can build your successor out of more bytes.
Or, in popular culture terms, the last 30 seconds of this Age of Ultron clip[1].
What do you think training (and fine-tuning) does?
No LLM currently adapts to the tasks its given with an iteration cycle shorter than on the order of months (assuming your conversations serve as future training data; otherwise not at all).
No current LLM can digest its "experiences", form hypotheses (at least outside of being queried), run thought experiments, then actual experiments, and then update based on the outcome.
Not because it's fundamentally impossible (it might or might not be), but because we practically haven't built anything even remotely approaching that type of architecture.
But there is no reason the company can't come up with a different paradigm.
But I suppose you could say we don't know 100% since we don't fully understand how the brain learns.
1. Either you are correct and the neural networks humans have are exactly the same or very similar to the programs in the LLMs. Then it will be relatively easy to verify this - just scale one LLN to the human brain neuron count and supposedly it will acquire consciousness and start rapidly learning and creating on its own without prompts.
2. Or what we call neural networks in the computer programs is radically different and or insufficient to create AI.
I'm leaning to the second option, just from the very high level and rudimentary reading about current projects. Can be wrong of course. But I have yet to see any paper that refutes option 2, so it means that it is still possible.
If you wanted to reduce it down, I would say there are two possibilities:
1. Our understanding of Neurel Nets is currently sufficient to recreate intelligence, consciousness, or what have you
2. We’re lacking some understanding critical to intelligence/conciousness.
Given that with a mediocre math education and a week you could pretty completely understand all of the math that goes into these neurel nets, I really hope there’s some understand we don’t yet have
MLPs and transformers are ultimately theoretically equivalent. That means there is an MLP that represent the any function a given transformer can. However, that MLP is hard to identify and train.
Also the transformer contains MLPs as well...
(I did AI and Psychology at degree level, I understand there are definitely also big differences too, like hormones and biological neurones being very async)
Transformers, while not exactly functions, don't have a feedback mechanism similar to e.g. the cortical algorithm or any other neuronal structure I'm aware of. In general, the ML field is less concerned with replicating neural mechanisms than following the objective gradient.
Numenta has attempted to implement a system to this effect (see the wiki page https://en.wikipedia.org/wiki/Hierarchical_temporal_memory) for quite some time with not particularly much success.
Personally I think the kinds of minds we create in silico will end up being very different, because the advantages and disadvantages of the medium are just very different; for example, having a much stronger central processor and much weaker distributed memory, along with specialized precise circuits in addition to probabilistic ones.
1. Humans have general intelligence. 2. Human brains use biological neurons. 3. Human biological neurons give rise to human general intelligence. 4. Artificial neural networks (ANNs) are similar to human brains. 5. Therefore an ANN could give rise to artificial general intelligence.
Many people are objecting to #4 here. However in writing this out, I think #3 is suspect as well: many animals who do not have general intelligence have biologically identical neurons, and although they have clear structural differences with humans, we don’t know how that leads to general intelligence.
We could also criticize #1 as well, since human brains are pretty bad at certain things like memorization or calculation. Therefore if we built an ANN with only human capabilities it should also have those weaknesses.
They don't, actually.
A very cynical take is that this is an extreme version of 'we plan to spend all money on growth and figure out monetization later' model that many social media companies with a burn rate of billions of $$, but no business model, have used.
He is saying he will try to build something head and shoulders above anything else, and he got a billion dollars to do it with no expectation of revenue until his product is ready. The likelihood that he fails is very high, but his backers are willing to bet on that.
i read the article but I am not sure how they know when this condition will be true.
Is this obvious to ppl reading this article? is it emperor has no clothes type situation ?
Are these ppl merely gullible or coconspirators in the scam ?
If you check the 2024 YC batch, you'll notice pretty much every single one of them mentions AI in some form or another. I guarantee you the large majority of them are just looking to be bought out by some megacorp, because it's free money right now.
1b was a non profit donation, so there wasn't an expectation of returns on that one.
There's plenty of players going for the same goal. R&D is wildly expensive. No guarantee they'll reach the goal, first or even at all.
Moreover, the majority of the capital likely goes into GPU hardware and/or opex, which VCs have currently arbitraged themselves [3], so to some extent this is VCs literally paying themselves to pay off their own hardware bet.
While hints of the ambition of the Manhattan project might be there, the economics really are not.
[1a] https://www.getpin.xyz/post/clubhouse-lessons-for-investors [1b] https://www.theverge.com/2023/4/27/23701144/clubhouse-layoff... [3] https://observer.com/2024/07/andreessen-horowitz-stocking-ai...
No. The Manhattan Project started after we understood the basic mechanism of runaway fission reactions. The funding was mostly spent purifying uranium.
AGI would be similar if we understood the mechanism of creating general intelligence and just needed to scale it up. But there are still fundamental questions we still aren’t close to understanding for AGI.
A more apt comparison today is probably something like fusion reactors although progress has been slow there too. We know how fusion works in theory. We have done it before (thermonuclear weapons). There are sub-problems we need to solve, but people are working on them. For AGI we don’t even know what the sub-problems are yet.