I've always thought it's never too early to start allocating significant resources to AGI research and safety given the potential impact. That said, I've up until very recently agreed with your take on the situation.
What changed my mind was an article detailing the latest advances in silver nanowire mesh networks.[0]
I knew neural computing was a thing, but not that we already had a computing substrate capable of self-organizing its own neural architecture based entirely on external input, with power requirements analogous to the human brain. No firmware or software required.
One could say that human physiology remains far more complex just on the substrate front alone, what with the brain being an incredibly complex, delicate balance of chemicals and heterogeneous cells. However, this particular artificial substrate is already succeeding in basic learning tasks, despite the fact it's far simpler.
I steongly suspect we've figured out at least one artificial computing substrate not only capable of, but perhaps well-suited towards producing AGI, and that it's just a matter of scaling it.
Of course, once you scale the technology sufficiently, the question then becomes how to architect and train it into an AGI. You say as much above, but I suspect the architecture need not be human to be a threat, or to otherwise become extremely powerful.
[0] https://www.quantamagazine.org/a-brain-built-from-atomic-swi...
> the question then becomes how to architect and train it into an AGI.
If that’s the question then why even bother with silver nanowire “brains”? Why not just grow a human brain out of some suitable stem cells and work with that? Leaving aside the massive creepiness factor.
As messed up as it is, using biology as you describe might actually be safer once we get close to AGI, even if it is creepy as hell. There's something to be said for having a machine that's physically limited to thinking as fast as a human can (even if it is vastly smarter), versus one that can figure out a new branch of physics in the time it takes to blink.
I’m wondering if it’s possible to code a simulation of what these wires are doing? Since most of us don’t have access to the nanoscale silver contraption we could still be studying the operation.
He asked a panel for the least impressive thing they did not believe would be possible within a few years. In other words, pick the point closest to the boundary of that classifier. Obviously my future knowledge is imperfect, and anything close to the boundary is subject to a lot of uncertainty. From that difficulty, he hand waves an argument that long term prediction of the unlikelihood of AGI is folly.
The problem is that these aren't in the same class of predictions. One is detailed and precise; the other coarse and broad. Predicting that it will rain at 2:00 PM November 10, 2017 is much more difficult than predicting that the average summer of 2040-2060 will be hotter than the average from 1980-2000. Precise local predictions just arent the same thing as broad global predictions, and difficulty doesn't transfer, because I'm not bootstrapping my global prediction on the local one. I'm using different methods entirely.
There's a similar thing with AI, I think. I can't confidently tell you what the big splash we'll see at NIPS next year or the year after. But I can look at the way we know how to do AI and say I don't think 30 years will see a machine that can make dinner by gathering ingredients from a supermarket, driving home, and preparing the meal.
Really? Why not? Once or twice, if we cherry-pick its performance, or reliably?
This is really surprising to me.
We might be able to make machines to do each of those tasks, but that's not the answer. I might do 100,000 things in an average week. Clearly we aren't going to build 100,000 bespoke CNNs and LSTMs. To worry about superhuman AI, we probably have to figure out how to make one or a few machines that aren't gloried deep fryers.
I get what you mean, but I don't think we should assume this.
And it will be a great boon. Quality of meals will go up and costs will go down. The restaurant market will shrink but not completely disappear.
But McDonald's will certainly die, as there'll be no need to sacrifice quality and nutrition to get speed and convenience. In fact, a table at McDonald's will be an inconvenient booth.
Yes, it is much easier to make predictions about the far future which no one will remember or care about when the time comes to test there veracity.
That does not make them more accurate.
Your "will it rain" example is a good one, but it's easy to counter - I can't say what the world map will look like exactly tomorrow, but it will be a hell of a lot better than even my coarse prediction of the world map in 2040. I think.
It's also true that it doesn't follow from "short-term prediction of x is hard" that "long-term prediction of y is harder". But there must be short-term patterns, trends, or observable generalizations of some kind that you're incredibly confident of, if you're even moderately confident about how those patterns will result in outcomes decades down the line, and if you're confident that the things you aren't accounting for will cancel out and be irrelevant to your final forecast. (Rather than multiplying over time so that your forecast gets less and less accurate as more surprising events chain together into the future.)
If those ground-level patterns aren't a confident understanding of when different weaker AI benchmarks will/won't be hit, then there should be a different set of patterns confident forecasters can point to that underlie their predictions. I think you'd need to be able to show a basically unparalleled genius for spotting and extrapolating from historical trends in the development of similar technologies, or general trends in economic or scientific productivity.
I think Eliezer's skepticism is partly coming from Phil Tetlock's research on expert forecasting. Quoting Superforecasting:
> Taleb, Kahneman, and I agree that there is no evidence that geopolitical or economic forecasters can predict anything ten years out beyond the excruciatingly obvious – ‘there will be conflicts’ – and the odd lucky hits that are inevitable whenever lots of forecasters make lots of forecasts. These limits on predictability are the predictable results of the butterfly dynamics of nonlinear systems. In my EPJ research, the accuracy of expert predictions declined toward chance five years out. And yet, this sort of forecasting is common, even within institutions that should know better.
So while we can't rule out that making long-term predictions in AI is much easier than in other fields, there should be a strong presumption against that claim unless some kind of relevant extraordinarily rare gift for super-superprediction is shown somewhere or other. Like, I don't think it's impossible to make long-term predictions at all, but I think these generally need to be straightforward implications of really rock-solid general theories (e.g., in physics), not guesses about complicated social phenomena like 'when will such-and-such research community solve this hard engineering problem?' or 'when will such-and-such nation next go to war?'
Great point, I'll be stealing that one ;)
One annoyance I have heard about SV is that all the companies are just trying to replace your Jewish Mom: Uber/Lyft is Mom's minivan, GrubHub/DoorDash/BlueKitchen is Mom's cooking, Google is Mom's encylopedia, Yelp is the synagogue's meeting hallway, Tinder is your Mom's yenta, etc. The examples abound in a non-B2B space.
In that vein, then AGI is not just a Superman fallacy, but SuperMom too.
I think AGI is likely closer to the present than 1987 was -- that is, I'd bet on having AGI by 2047. (Note: this is distinct from superhuman AGI.) Do you not agree?
I think a lot of people underestimate NNs because they think of NNs in terms of the semantics of their history instead of all possible semantics that can be fit to tensor networks. We know [P] that NNs are a sufficient abstraction to model human intelligence if we had arbitrary compute -- the questions that remain are all about making the hardware faster enough and the estimators efficient enough (which may require moving off tensor networks, but it's still only a refinement of the mathematics used).
Of course, one could argue that humans are caught in a "tensor trap", in that too much of our intellectual effort is now relying on estimators built out of networks of tensors. (I do.) But even then, AGI is likely to appear out of similar methods with new mathematical objects.
[P] Proof NNs can compute human intelligence with arbitrary compute:
You can embed the standard model as a NN by changing how you view the network of tensor equations. Human intelligence is (arguably) embeded in the standard model by modern science. So we can embed a model of human intelligence in a (large enough) NN.
This isn't immediately computationally useful, but it shows that there's not a fundamental flaw in using an estimator built out of a DAG of calculations to model intelligence if we can find an appropriate estimator for our computational needs.
Not sensibly in terms of years, no.
It's more a handwaving gut feeling combined with an intuition.
I didn't feel like there was a royal road from symbolic AI to AGI. It doesn't feel to me like there is one from NNs either.
As for the intuition: perhaps it's because it was my PhD topic, I have always felt that there needs to be a breakthrough in emergence, specifically in evolutionary computing (or some other system in which there is a tight feedback loop between behaviour and survival). Something to unshackle the development of AI from human beings deciding what behaviours they want to engineer.
The resulting computation would be orders of magnitude less efficient, considerably less understandable (it is very unlikely to wash dishes or write poetry), but crucially much less fragile. And it has always been the fragility of the engineering which has made AI feel a little smoke and mirrors at times. NN are massively less fragile than symbolic systems, (and orders of magnitude less efficient, for problems symbolic systems are good at) but it does feel like we need another fundamental step.
But, my feelings aside, I agree with the article because I recognise this could well be a 'Manhattan Project' type of event.
I've got a repo[2] for a vm where the programs can act in that way. But I am at the early stages of programming initial programs with economic/learning strategies. So I don't know how promising it is. More details can be found spread out on my blog [3]
[1] https://e-drexler.com/d/09/00/AgoricsPapers/agoricpapers.htm...
[2] https://github.com/eb4890/agorint/
[3] https://improvingautonomy.wordpress.com/2017/08/13/introduct...
For that, a technique must be able to compute human intelligence at a useful speed on a constructable system.
Using a network of tensors to compute intelligence is incredibly old (I believe, dating back about 80 years), but has only recently become tractable to do for any complex tasks.
However, in the past ~30 years, we've gone from "intractable for moderate problems" to "world champion at go", "able to detect cancer in images as well as experts", etc. My contention is that in another ~30 years, we'll see a step sufficient for "can do average at most intellectual activities", even if that's just having the storage to keep 10,000 task specific NNs (of AlphaGo sophistication) on hand to interpolate all actions as mixes of specialist tasks. Do you really not think there's a strong heuristic case for that? (I would contend that you should be able to point to a specific task you don't think it will be able to do on that timeline -- do you know of such a task?)
The proof was merely that we're not barking up a theoretically dead tree -- we have to rely on heuristics for if it will eventually converge to tractable.
Not unless there was good reason to expect that every other possible application of NNs would be as difficult to achieve as general intelligence.
The standard model could be computed directly without NNs, which I think we agree wouldn't be a useful way to approach AGI.
Feedback and memory are really important features of GI that you will not get out of a DAG ever. You need loops for that.
All loops can be modeled as a DAG and single attached piece of memory (of sufficient width) allowed to execute to a steady state; sorry if it wasn't clear that I was talking about things like NTMs too. (It's why I used 'tensor network' most places; also, in practice, we tend to let subgraphs reach a steady state independently where possible.)
Your comment is also an excellent example of a strawman: you picked out the word 'DAG' to raise a technical argument when the usage of DAG versus general tensor networks clearly wasn't the main point (as some NNs have feedback and the standard model is posed as differential equations).
It's more constructive to respond to the strongest point, not pick at technical details that can easily be rephrased.
But I find myself agreeing with this article. Strongly.
And I have long suspected that, we miss a lot of the significance and opportunities in AI, because we have only one exemplar of 'higher' intelligence: a human being. AI folk are so concerned with getting computers to do the things that humans are good at, I suspect most will miss / 'refute' / deride the inflection point, because the system can't wash the dishes (or some other form of embodied cognition), or write poetry humans would find beautiful, (or understand some other socially-conditioned cue).
The superhuman fallacy really is the bane of AI.