News for hackers. Stuff that matters.
Re:Concrete AI Safety Problems by Mendenhall on Tuesday June 21, 2016 @09:33PM (#11950986)
I always get the feeling AI is going to be like nuclear capability. Great reason to create it, but then once its made everyone wants to get rid of it.
Re:Concrete AI Safety Problems by JoshTriplett on Tuesday June 21, 2016 @09:47PM (#11951034)
There's a huge difference, though. Creating a nuclear weapon encourages others to do the same. An AGI, if done right, would never allow the creation of a second one with conflicting values; there should be no second AGI.
Re:Concrete AI Safety Problems by argonaut on Wednesday June 22, 2016 @02:52AM (#11952018)
You are mistaking an AGI for an artificial superintelligence (I might also add that the very concept of superintelligence is pure speculation - basic AGI at least can be grounded in replication of human brains). The first AGI will be closer to a low-IQ human than a Machiavellian super-optimizer.
Re:Concrete AI Safety Problems by xyience on Wednesday June 22, 2016 @09:28AM (#11953695)
I don't think there's much reason to be confident that the first AGI will be like a low-IQ human, at least for very long, even if it starts off as an emulated human brain. Machines have the huge advantage of faster materials (neurons are slow), perfect memory, perfect calculation, ability to scale horizontally, backups to restore in the event of bad self-modification experiments, and no need for things like sleep and food to take up its time from learning, improving itself, and acting upon the world.
Re:Concrete AI Safety Problems by argonaut on Wednesday June 22, 2016 @04:46PM (#11956864)
You've omitted the huge, overwhelmingly outweighing disadvantages of intelligence in machines, which is that they don't work. Given that research progress is incremental, there really isn't reason to believe we will jump from narrow AI to super-smart AGI, instead of narrow AI to dumb AGI to smart AGI to super-smart AGI.
Re:Concrete AI Safety Problems by xyience on Wednesday June 22, 2016 @05:16PM (#11957091)
There are historical examples of discontinuities, though I don't think the FOOM debate will be settled soon. It may be quite a while before we even get to "dumb AGI", but the hardest part of that is the G part. Right now "they don't work" is indistinguishable from "they don't exist", but if we get that G, I don't see how you could claim either. From there, even if we suppose it's another huge leap to get to true super-intelligence instead of a FOOM, the time to get to merely smart AGI, and indeed smarter-than-human AGI, would be short, if only for the basic advantages of a silicon machine substrate. If all we had were human brains running on silicon even, that would be enough to quickly reach superhuman general intelligence, even if not true super- (or perhaps ultra- as I.G. Good originally put it) intelligence that we expect for a Singularity event.
Re:Concrete AI Safety Problems by argonaut on Wednesday June 22, 2016 @07:02PM (#11957738)
This makes no sense. This is just unsubstantiated speculation right now. There is no reason to believe that dumb AGI will hardware-scale to smart AGI for free. In fact most machine learning algorithms have diminishing (logarithmic) returns with data and compute.
Re:Concrete AI Safety Problems by JoshTriplett on Wednesday June 22, 2016 @11:18AM (#11954592)
> I might also add that the very concept of superintelligence is pure speculation - basic AGI at least can be grounded in replication of human brains

I'd tend to think the reverse: the idea that we can recreate general intelligence requires some degree of speculation, but once we have general intelligence running on a computer, it takes a fairly contrived set of conditions for it to not scale up exponentially.

Re:Concrete AI Safety Problems by argonaut on Wednesday June 22, 2016 @04:44PM (#11956850)
This is speculation. We don't even know what general intelligence on a computer will look like, therefore we don't know what constraints there will be, therefore it is speculative to say there won't be constraints on exponential scaling.
Re:Concrete AI Safety Problems by otoburb on Tuesday June 21, 2016 @10:06PM (#11951086)
Might never want to allow, but may not necessarily have a choice in the matter. Nobody really knows how singularity AGI fooms will play out.
Re:Concrete AI Safety Problems by JoshTriplett on Tuesday June 21, 2016 @10:18PM (#11951120)
> Might never want to allow, but may not necessarily have a choice in the matter.

"if done right" glosses over quite a bit, yes.

Re:Concrete AI Safety Problems by daveguy on Tuesday June 21, 2016 @10:14PM (#11951110)
Probably the same way flying cars played out.
Re:Concrete AI Safety Problems by pixl97 on Tuesday June 21, 2016 @10:54PM (#11951251)
Huh? We have flying cars. They are called planes, and they take a lot of maintenance to keep from falling out of the sky.

But let's change the question up a bit... There are billions and billions of flying intelligences on this planet. Birds, insects, even mammals. Nature has already created that. We've created things that are even better at flying fast and carrying more weight. So simply looking at 'flying cars' and saying they didn't happen so AI can't happen is at the least, very ignorant.

If nature can create something randomly, we can create something directed in a shorter period of time (well, we don't really have another 4 billion years to try). AGI is an eventuality.

Re:Concrete AI Safety Problems by argonaut on Wednesday June 22, 2016 @12:17AM (#11951515)
Saying AGI is an eventuality is as speculative as saying an alien invasion is an eventuality.

We have evidence of intelligent species (ourselves) invading. We also have evidence spaceflight is possible.

Re:Concrete AI Safety Problems by pixl97 on Friday June 24, 2016 @04:15PM (#11973092)
GI already exists, AGI is only speculative if you believe that humans go extinct before they get complex. We can perform scientific experiments on GI at this time.

An alien invasion is speculative at this point because there is no alien species on to which we can base any scientific refutation. It is not impossible, it is simply impossible to define any probably of occurrence.

Re:Concrete AI Safety Problems by onion2k on Wednesday June 22, 2016 @02:01AM (#11951853)
Whether or not an AGI should have the capability to stop the creation of a second AGI is exactly the problem the article is considering.