News for hackers. Stuff that matters.
Re:Concrete AI Safety Problems by lacker on Tuesday June 21, 2016 @09:12PM (#11950901)
This sort of "internal" approach to AI safety, where you attempt to build fundamental limits into the AI itself, seems like it is easily thwarted by someone who intentionally builds an AI without these safety mechanisms. As long as AI is an open technology there will always be some criminals who just want to see the world burn.

IMO, a better approach to AI safety research is to focus on securing the first channels that a malicious AI would be likely to exploit. Like spam, and security. Can you make communications spam-resistant? Can you make an unhackable internet service?

Those seem hard, but more plausible than the "Watch out for paperclip optimizers" approach to AI safety. It just feels like inventing a way to build a nuclear weapon that can't actually explode, and then hoping the problem of nuclear war is solved.

Re:Concrete AI Safety Problems by jimrandomh on Tuesday June 21, 2016 @09:19PM (#11950931)
For the most part these aren't aimed at people making bad AGIs deliberately, but rather, at well-intentioned developers launching AGIs with serious bugs. Those developers will want to include safety mechanisms, and will, hopefully, be the first to make AGIs with major capabilities.

We should also be working to secure the channels an AGI might exploit, but most of those are already tied to an economic incentive to invest in security, and are already getting large investments compared to the relatively tiny field of AI safety.

Re:Concrete AI Safety Problems by lacker on Wednesday June 22, 2016 @12:52AM (#11951635)
These aren't aimed at people making bad AGIs deliberately, but rather, at well-intentioned developers launching AGIs with serious bugs.

That is precisely the approach that does not make sense to me. By the time friendly developers can launch AGIs, unfriendly developers will not be far behind. And AI safety seems likely to be an issue far before the development of AGI - a non-general malicious AI that can only hack internet services or trick humans into running arbitrary code via conversation is already quite a serious problem.

So to me focusing on "unfriendly developers building narrow AIs" seems more logical than focusing on "friendly developers building AGI".

Re:Concrete AI Safety Problems by Hondor on Wednesday June 22, 2016 @01:09AM (#11951691)
People can already build robots with guns that run around shooting people, but they usually don't. We still have laws to protect us from that. Nation states have the resources to build armies of gun-shooting robots and they already do! But again, they don't use them to destroy humanity for other reasons, like politics and retaliation and all that.

So I don't think we need to worry about someone maliciously making an evil robot. That's already a problem and we already spent thousands of years figuring out systems of society to protect us from those dangers.

Re:Concrete AI Safety Problems by jdhendrickson on Wednesday June 22, 2016 @05:18AM (#11952465)
I think without visiting the root of the site many people are afraid these are ground rules for strong AI development as an open source project. If that were the case with Google's backing I would be more than apprehensive.

We all know strong AI wouldn't be some sort of robot running around shooting people like a movie, this would be an extinction event worse than a rather large asteroid if it fell into the wrong hands.

The first country or corporation to create strong AI capable of self evolution will either be able to immediately take control of the world as we know it, or worse create something capable of destroying humanity as we know it.

That's not what's going on at all for anyone else that at first glance though that (I'll admit I am guilty of thinking that after glancing at the article as well).

Goal 1: Measure our progress Goal 2: Build a household robot Goal 3: Build an agent with useful natural language understanding Goal 4: Solve a wide variety of games using a single agent

That's all they are trying to accomplish with this (so far anyway). None of which requires strong AI.

TLDR they are formulating some laws of robotics that are a bit more granular for dumb AI which are incapable of self improvement and very task oriented.

Re:Concrete AI Safety Problems by Retric on Wednesday June 22, 2016 @10:56AM (#11954398)
You are thinking in narrative terms about AI. Crazy <-> Rational is a far more likely issue with early AI than Good <-> Evil. The problem is you can't optimize for general intelligence only solutions to given problems. Let's say you have two twins one of which is smarter than the other. If you ask about future predictions then how do you know the correct answer before hand.

In other words the 99% of paperclip optimizers are going to start making virtual paperclips not paving over the universe with actual paperclips. Hacking your reward function is easer than solving hard problems.

PS: Don't do drugs :0

Re:Concrete AI Safety Problems by ajuc on Wednesday June 22, 2016 @08:47AM (#11953422)
> By the time friendly developers can launch AGIs, unfriendly developers will not be far behind.

I think making friendly or unfriendly AI is similarily hard (you depend on the same variables just with different THEN clauses).

Making accidentally homicidal well-meaning AI is much easier than both IMHO.

Re:Concrete AI Safety Problems by argonaut on Wednesday June 22, 2016 @02:55AM (#11952027)
The solution is this: call the police and arrest the malicious developers.
Re:Concrete AI Safety Problems by daveguy on Tuesday June 21, 2016 @10:05PM (#11951085)
You are right, except most of these are applicable to narrow AI and don't need to wait for AGI for their application.
Re:Concrete AI Safety Problems by ikeboy on Tuesday June 21, 2016 @09:23PM (#11950952)
>All else being equal, not many people would prefer to destroy the world. Even faceless corporations, meddling governments, reckless scientists, and other agents of doom require a world in which to achieve their goals of profit, order, tenure, or other villainies. If our extinction proceeds slowly enough to allow a moment of horrified realization, the doers of the deed will likely be quite taken aback on realizing that they have actually destroyed the world. Therefore I suggest that if the Earth is destroyed, it will probably be by mistake.

https://intelligence.org/files/CognitiveBiases.pdf

Re:Concrete AI Safety Problems by lacker on Wednesday June 22, 2016 @12:43AM (#11951599)
I agree. But, so far we have come closest to destroying the world when nation states are stockpiling weapons of war, threatening mutually assured destruction, with their arsenals kept on a hair trigger. To me, that seems likely to keep being the main danger scenario in the future.

It would still be a "mistake", but not the "oops what does this button do" sort of mistake, but rather the "I could have sworn they fired first" sort of mistake.

That is why I think the main goal of AI safety should be how to defend against intentionally malicious AI. We should be thinking about drone swarms and hacker-AIs rather than paper clips.

Re:Concrete AI Safety Problems by Retric on Wednesday June 22, 2016 @11:04AM (#11954473)
One nation already went nuclear > non nuclear demonstrating a level of basic competence.
Re:Concrete AI Safety Problems by 21 on Tuesday June 21, 2016 @09:20PM (#11950935)
This paper seems more concerned with near-future and non-malicious AI safety - ie: your self driving car not running off a cliff or your rumba not spewing it's dust bin all over so that it has new work to do.

The stuff you talk about is farther in the future, and a much tougher problem.

Re:Concrete AI Safety Problems by arcanus on Tuesday June 21, 2016 @09:33PM (#11950984)
Excellent post, I rather agree.

> As long as AI is an open technology there will always be some criminals who just want to see the world burn.

Thankfully we have yet to see terror attacks with nuclear weapons, but the lower barriers of entry for potentially catastrophic AIs undoubtedly is alarming.

I often wonder if analog solutions will not be critical. To continue with your example, a nuke that must be armed with a physical lever would be a rather great hindrance to any AI. Internet communications, etc. will be more tricky, due to the high frequency activity, but the point of having meatspace 'firewalls' on mission critical activities is a nice short term kludge.

Re:Concrete AI Safety Problems by daveguy on Tuesday June 21, 2016 @10:12PM (#11951102)
Low barrier to entry for catastrophic AI? Surely you aren't talking about the current state of AI. The current barrier of entry for an AGI (which is at least what it would take for catastrophe) is "does not exist". That's about as big as barriers get. Even when the first AGIs show up it will take full time supercomputers to perform like a single human brain. I don't see that as a particularly low barrier.
Re:Concrete AI Safety Problems by taneq on Tuesday June 21, 2016 @11:20PM (#11951338)
This is making the assumption that AGI is computationally expensive rather than just requiring of a particular algorithmic approach. It may be possible (and in fact I'd expect it to be so) to replicate general human-level intelligence with significantly less raw computational power than that embodied by the human brain.

Evolution does generate highly optimized systems but generally only when those systems have been around for tens of millions of years. Human-level intelligence has only been around for what, 50k - 100k years? We're probably still more in the 'just works' phase rather than the 'streamlined and optimal' phase.

Re:Concrete AI Safety Problems by pixl97 on Tuesday June 21, 2016 @10:44PM (#11951226)
Eh. The barrier for entry for an AGI does exist, though it is currently undefined, since we don't know what it is. The reason I say that is there are at least 7 billion general intelligences running around this planet (and many more if you consider animal intelligences) It is important to define it that way, not that it is impossible, just that it is unknown how much effort is needed to create an artificial one.

This distinction is very important when comparing the threat of AI with other significant threats. Before nuclear bombs were built we could not tell you what the difficulty was in creating one. Now that difficulty is well defined, and we can use that knowledge to prevent them from being built by most nations, except the most well funded.

If the barrier for entry for AGI (then ASI) is lower than we expect, then the threat of AI is significantly different than if AGI/ASI can only be created by nation states.

Re:Concrete AI Safety Problems by argonaut on Wednesday June 22, 2016 @07:08AM (#11952838)
The barrier for entry for an alien invasion does exist, though it is currently undefined, since we don't know what it is. The reason I say that is there are at least 1 bloodthirsty species running around this galaxy (and many more if you consider the statistical possibility of life on other planets) It is important to define it that way, not that it is impossible, just that it is unknown how much time is needed before an alien invasion.

The reason I am framing things this way is we need to be very careful here because we are starting to turn towards speculation.

Re:Concrete AI Safety Problems by FeepingCreature on Wednesday June 22, 2016 @10:07AM (#11954002)
You know, you mean for that to sound implausible, but the Great Filter is in fact an open research problem.
Re:Concrete AI Safety Problems by argonaut on Wednesday June 22, 2016 @04:42PM (#11956843)
I'm pointing out that this is all speculative and dangerously close to science-fiction.
Re:Concrete AI Safety Problems by pixl97 on Thursday June 23, 2016 @10:18AM (#11961013)
You should learn the difference between what is impossible and what just has not happened yet. Much science-fiction that was in the realm of possibility is now science-reality. One should not need reminded they are communicating at the speed of light over a global communications network capable of reaching billions of people at a time. I'm sure at one point in the past that was science-fiction, now reality. I don't believe you can show me any science that points out why AI/AGI/ASI can be created, we simply are not at that level of sophistication.
Re:Concrete AI Safety Problems by argonaut on Thursday June 23, 2016 @07:34PM (#11965089)
Your argument is basically "some science-fiction has sometimes turned out to be true." That doesn't counter the fact that this is just speculation.
Re:Concrete AI Safety Problems by pixl97 on Friday June 24, 2016 @04:11PM (#11973059)
Um, pretty much, no.

Science fiction turns out to be true when physical reality agrees that it can be true. This again, is why we have a global communications network and personal wireless devices connected to it. This is also the reason we do not go faster than light.

The reason we don't have flying cars is they are completely possible. They are also terribly dangerous and expensive and a complete waste of energy.

The reason we don't have AGI is not that it is impossible, again if nature can create it, we can recreate it. Since we don't have a good understanding of the networked nature of emergent intelligence we cannot create a power optimized network that would allow us to create a energy efficient version. AGI itself is a complete waste of energy at this point. We already have many types of AI that are energy efficient and used in products now.

Re:Concrete AI Safety Problems by argonaut on Saturday June 25, 2016 @12:12AM (#11975184)
> Science fiction turns out to be true when physical reality agrees that it can be true

This is a ridiculous argument. Furthermore, even if it were true, it tells us nothing about the timeline. It could take 10,000 years for all we know.

Re:Concrete AI Safety Problems by tlb on Tuesday June 21, 2016 @10:51PM (#11951242)
In the past, single human brains have come close to destroying the world, and lots of people have access to supercomputers, so the barrier doesn't seem insurmountable.

I don't think you need AGI to cause a catastrophe. A narrow AI specializing in cyberattacks could be catastrophic, and is probably possible with current techniques.

Re:Concrete AI Safety Problems by ccvannorman on Tuesday June 21, 2016 @09:43PM (#11951020)
one of the most effective and scariest attack vectors for AI would be convincing or coercing humans to do its bidding (e.g. "pull the lever or i'll switch off your father's life support")
Re:Concrete AI Safety Problems by deftnerd on Tuesday June 21, 2016 @10:50PM (#11951237)
It might be easier than that. Data Broker companies are the ones that run those stupid "Which LOTR character are you" or "What color are you" quizzes that are popular in some social media circles. They do it to slowly build psychological trait models of the people taking the quiz. This allows them to sell that information to marketing companies.

Access to that kind of data would help an AI determine the people that are more susceptible to manipulation. Add in records on health care, as you mentioned, and information on debts and you have data that can help an AI gather as many human minions as it needs.

Re:Concrete AI Safety Problems by pixl97 on Tuesday June 21, 2016 @10:56PM (#11951254)
Greedy people cause all kinds of problems now. I don't know why people aren't concerned that a greedy AI could be even worse.
Re:Concrete AI Safety Problems by kbaker on Tuesday June 21, 2016 @11:36PM (#11951395)
Great recent story following this plot line: http://compellingsciencefiction.com/stories/crinkles.html
Re:Concrete AI Safety Problems by ikeboy on Wednesday June 22, 2016 @08:45AM (#11953409)
Re:Concrete AI Safety Problems by simonh on Wednesday June 22, 2016 @05:20AM (#11952482)
We don't have to choose between making inherently safe AIs and creating and environment safe from hostile AIs. We should do both. Apart from anything else, we already have a planet full of intelligent agents, many of which are highly hostile, so increasing our general level of security should already be a high priority.
Re:Concrete AI Safety Problems by pizza on Tuesday June 21, 2016 @11:42PM (#11951414)
"Defenders think in lists, attackers think in graphs" redux
Re:Concrete AI Safety Problems by xyience on Wednesday June 22, 2016 @09:36AM (#11953763)
Can you make an unhackable person? If people can get around your safeguards, even only people with authority rather than in general, the AGI can too. Trying to keep a potentially malicious (actively or passively so, I mean only bad for humans) AGI in a box cannot work, we either get the internal parts right so we can be very confident it's not malicious to begin with (and so no need for a box attempt) or we fail.
Re:Concrete AI Safety Problems by Aelinsaar on Tuesday June 21, 2016 @09:40PM (#11951008)
It probably won't require maniacs out to destroy humanity, just a typically careless black market.
Re:Concrete AI Safety Problems by throwanem on Wednesday June 22, 2016 @11:57AM (#11954874)
> Can you make communications spam-resistant? Can you make an unhackable internet service?

Maybe. Can you make a human that can't be suborned by a superhuman evil AI?

Re:Concrete AI Safety Problems by ajuc on Wednesday June 22, 2016 @08:44AM (#11953407)
> It just feels like inventing a way to build a nuclear weapon that can't actually explode [accidentally], and then hoping the problem of nuclear war is solved.

That's how cold war remained cold.