like everything, it will take much more resources and time than we can predict today with our best estimates, partly because that's just how these things turn out, and mostly because a true AGI will likely require billions of neural networks all adapting and swapping neurons and communications pathways amongst themselves and training themselves at the same time they are doing useful work.
we have no idea how to do anything like this at scale, yet.
Did you know what is the difference between the military simulators used by defense departments and strategy games like Go? Did you know both share a lot in common? What you dismiss as "game playing" has more present applications than you imagine and will make the difference between life and death in armed conflicts.
The orders given by generals will likely come from a successor of MuZero. And that may expand to every key decision making process, like the equipment to be manufactured, logistics, etc.
Combine that with the power asymmetry coming from drones and that "game playing" will become the most dominant force in this world.
There's much that can be done to trick the opponent into making incorrect assumptions.
I know next to nothing about game theory, but aren’t games with perfect information what would generally be considered “fair”?
The argument is compelling to me and fits my experiences. I think the limitations of these tools has become more apparent to non-specialists the last couple years, so if he's been saying this since 2016 that's honestly more impressive. What do you expect him to change his tune to if he's already singing a good one?
In mathematics, you solve problems from both directions, at times.
It seems that computer science has come full circle with psychology, finally.
Dig into computational comparative psychology for leads on where deep learning experts need to turn to next.
There are plenty of computational neuroscientists that are probably struggling with untwining what they are looking at.
Arguably, it is the only path to strong AI (if you believe that it is even possible).
It just doesn't scale.
On what planet? I can give you about 7 billion reasons why it isn't this one.
9 women do not a baby in 1 month make. Once you have said bab(y|ies), you then have the inconvenience of them having a will of their own, you have to support and eventually pay them, and dear god, make too many of them and they might even just throw you away instead of going in the direction you want.
That kind of not scaling.
I mean... If no one else is willing to say it, I'll call a spade a spade; because that is precisely the major draw of the computer as ecpnomic/logistical lubricant. Doing jobs of people, but not getting paid or having to be accounted for as one.
The only thing that meets this definition of scaling is our fantasies. Absolutely everything else uses resources implying an opportunity cost and has non-zero latency in making any desired change.
How many of the computers we use to program transaction processors and other things are seperate from the entities that operate them? How many human actors were displaced, and salaries saved by cutting of jobs we've automated away?
We look at vehicle automation, machine learning, etc... Precisely because they open up new avenues through which money can be saved by not employing as many expensive bags of carbon and water in favor of throwing a handful of graphics cards, sensors, hard drives, and compute resources at problems that like it or not, we required fleets of humans for previously.
If it weren't in some fashion more economically viable at some level, in some way, I'm fairly sure we wouldn't still be pursuing it.
That's the thing I've noticed. It's the grim version of the more rosey "we're empowering fewer people to do more with less."
However, that doesn't make it impossible for a plan-producer to be both superhuman and non-sapient.
Nor do I think the default is for an AGI to be sapient.
(Though, possibly the default could be such that it isn't but in a way in which we couldn't be sufficiently confident that it isn't, and therefore be obligated to behave as if it is sapient)
But, in the FOOM scenario, I would expect that "whether or not to treat it as if it has rights" would be the least of our worries. (the greatest of our worries would be, roughly, "whether it kills everyone".)
Being capable of acting in general environments, planning and taking actions in the world, etc. I think probably aren't in themselves enough for the agent to have moral patienthood.
But, that's not to say that it wouldn't be eventually possible to make an AI which was a moral patient (like, suppose it was not just able to plan taking itself and its own planning capabilities into account, but was in addition, also sapient, in the sense of having an internal experience like that of our own. In this case (e.g. perhaps a computer emulation of a human mind) then I'm fairly confident that such an agent would be a moral patient. Such an agent being copyable as data would no more prevent them from being a moral patient than having a magic duplicator ray which could duplicate a living human including their body, would render all people not moral-patients, namely, not at all.)
edit: that's not to say that it wouldn't be catastrophic, but, having such a duplicator ray might also be catastrophic for the same reasons, and the presence of such a magical duplicator array shouldn't render all people suddenly not moral-patients, and so the copyability of AIs shouldn't by itself prevent them from being moral patients either.
I do think the duplicator ray is a great analogy. There would be a lot of tricky moral questions raised if such a technology existed.
On the topic of a human mind in a computer, i dont think they should be afforded property rights either. If you can copy them into 10 bodies, and each one can own a house, then they are taking real resources from actual humans. This extends to owning other things too.
But who is "they"? If you copied them 10 times, it's not like they act as a collective; there are 10 unique instances having unique experiences. What's wrong with 10 different people each owning a house?
Imagine if you had to singlehandedly compete with the productive output of a small nation to even stand a chance of competing...
> taking real resources from actual humans
I suspect these statements are contradictory. What is an "actual human", anyway? Next you'll be trying to tell me that entities without souls don't need rights. How do we detect souls? Well, you kind of just know.
An AGI which demonstrates sentience and its own independent wants/needs would raise substantial ethics questions if not afforded some rights.
Then I think you'd have to work your way down from there. We'd have to really ensure that these things have no capacity for learning, emotions or pain if we want to abuse these things with no ethical qualms.
I dont think this is as axiomatic as people think. Why should we give them rights? Should a program be able to own a house and displace a human? If they are effectively immortal, can continously learn, and consistently act logically, giving them human rights will eventually force humans into an underclass, because we dont have those advantages (i understand this is a bit hyperbolic).
If you are interested in equity at all between humans an AI, i think you have to take into account the advantages an ai might have and give humans a lot more rights to counteract those advantages.
>i think you have to take into account the advantages an ai might have
Yes. No disagreement there.
Not in any country with modern laws they don't. Murder is wrong.
The unsexy part of CS. The Should I vs the could I.
It's not reasonable to discount the training of parents/predecessors only when we're talking about machines; they still would not exist without them.
The difference here is that while karpierz's last step is, at least for now, the final one in a process producing a strong, generalized, self-awarely-conscious theory-of-mind-holding intelligent agent, the evolution of DL systems has not yet done so.
Well, there’s your scaling problem.
It should only take an instant to create a Strong AI (if you believe that Strong AI can be created).
What’s Weak about a Strong AI that takes a couple of years to make but pees on the floor all the time?
We have a pretty poor understanding of how the human brain (our only "Strong AI" evidence) learns, but the one approach that has been pretty definitively ruled out is large-scale backpropagation in the style that most deep learning pursues today. Maybe backprop is a true alternative approach to AGI, but it's hard to be completely convinced of this.
I haven’t finished reading it, but they make the case that if humans are conscious, then PDP is the way Strong AI will be figured out.
https://mitpress.mit.edu/books/parallel-distributed-processi...
I agree that there’s too much emphasis on the “robot”, but coming from an academic background in mathematics and psychology (and biology), I cannot take PDP as anything other than “this is how brains work”. Granted, half of PDP, the book, is a series of examples of “ways it might work” from 40 years ago or so, but the other half is about fundamentally tying the psychology to the brain through parallel distributed processing as a conceptual framework.
In that sense, I don’t think PDP can be refuted any more than token economies or political polling can be refuted.
Thoughts?
In the meantime, I will bookmark your thesis for perusal.
Is DL a reasonable path to strong AI? Probably not. DeepMind is making cool stuff in game playing, but it's still "dumb" AI.
But Marcus has cornered himself as a professional skeptic of deep learning. A voice which is useful only at times.