News for hackers. Stuff that matters.
Re:What you wanted to know about AI by AndrewKemendo on Monday March 16, 2015 @12:30PM (#9212401)
It doesn't attempt to engage with the arguments of Stuart Russell, Nick Bostrom, Eliezer Yudkowsky, and others who are worried, not about current methods, but about what will happen when the time comes -- in ten, fifty, or a hundred years -- that AI does exceed general human intelligence.

That's because it would be arguing a straw-man. So there is no reason to engage the argument.

Re:What you wanted to know about AI by davmre on Monday March 16, 2015 @12:47PM (#9212549)
I'm not sure I understand what you mean by "straw man" here. The usual meaning is that to attack a straw man means to argue against a position that no one actually holds, but which is easier to attack than your opponent's actual position. The concerns about the long-term future of AI are real, actual beliefs held by serious people. At this point there's a respectable literature on the potential dangers of unconstrained, powerful optimizing agents; I linked a couple of examples. These arguments are well thought through and worth engaging.

By contrast, this article and many others are implicitly arguing against an actual straw man position, the position of "let's shut down AI research before it kills us all in six months", which no serious person on either side of this debate really holds (though it's understandable how someone who only read the discussion in the mainstream press could come to think this way).

Re:What you wanted to know about AI by AndrewKemendo on Monday March 16, 2015 @01:01PM (#9212667)
at this point there's a respectable literature on the potential dangers of unconstrained, powerful optimizing agents, of which I linked a couple of examples.

There isn't really though. Superintelligence, while an interesting book, doesn't do much more than pontificate on sci-fi futures based largely on writing from Yudkowski. Bostrom gives no clear pathway to AGI. Neither does Yudkowski in his own writings or Barrat (Our final invention). By the way Super intelligence is kind of an offshoot book from Global Catastrophic Risks.

All of them take these interesting approaches, WBE, BCI, AGI etc... and just assume they will achieve the goal without looking realistically about where the field is or how it would get there.

So what they do is they say: We are here right now. In some possible fantasy world we could be there. The problem is they can't draw the dots between them.

For example, find me someone who can tell me what kind of hardware we need for an AGI. Can't do it, because no one has any idea. What about interface, what is the interface for an AGI?

Even better (this was my thesis project): What group would have a strong enough requirement that an AGI would be required to build? Industry, Academia, Military? Ok great, can they get funding or resources for it?

etc...

Note, I am not saying that there is no potential that AGI could be a problem. The point here is that nobody has firm footing to say that it will definitely be a huge risk and that we need to regulate it's development.

There are actually people calling for legislation to prevent/slow AGI development - see Altman's blog post and many of the writings on MIRI. So that is what I am rallying against.

We need more development into AGI not less.

Re:What you wanted to know about AI by maaku on Monday March 16, 2015 @01:39PM (#9212925)
I'm more in your camp than MIRI's but...

Could you argue against the position directly? You kinda took a swerve in the middle there. Bostrom, Yudkowsky, and Barrat's positions are basically "superintelligence is possible, it will eventually happen in some way, and when it does ..." It's not within the scope of the quoted works to elaborate a technical roadmap or provide firm dates for the arrival of superhuman machine intelligence.

So you would like to see unencumbered research in this area. They argue that the long term outcome of this research is existentially risky and therefore should be tightly regulated, and the sooner the better. Do you have any points regarding this particular difference of opinion?

Re:What you wanted to know about AI by AndrewKemendo on Monday March 16, 2015 @02:38PM (#9213362)
Could you argue against the position directly?

No I can't because there isn't a way to argue it. I can't argue that it would be safe because I have nothing to point to that says it will be or even could be. We simply do not know enough about how it would be built to reason on it.

It's not within the scope of the quoted works to elaborate a technical roadmap or provide firm dates for the arrival of superhuman machine intelligence.

Correct, but they don't even reference a technical road map because there isn't one. It's too early to tell.

Do you have any points regarding this particular difference of opinion?

Not really, other than my own opinions which are as baseless as the others. I mean I have my own opinions and thoughts on things but they aren't empirically based at all, which is my main contention. We can't reason, let alone start making policy on stuff that we haven't the slightest idea how to build.

Re:What you wanted to know about AI by maaku on Monday March 16, 2015 @04:15PM (#9214065)
Well, color me disappointed. I think there are many arguments against the viewpoints of MIRI and Bostrom, both discrediting the perceived risk of super-intelligences and the the idea that we could build a "provably friendly" alternative.

But I don't think it's at all a fair criticism to say that you need a technical roadmap to engage in that debate. I would rather say that they need to be willing to discuss specific architectural details in order to even sit at the table, sure. The field of "AI safety" needs more engineering and less philosophy. But that doesn't require unity around a single roadmap, or any concept of deliverable dates.

Re:What you wanted to know about AI by AndrewKemendo on Monday March 16, 2015 @05:01PM (#9214364)
I would rather say that they need to be willing to discuss specific architectural details in order to even sit at the table, sure.

We are in violent agreement then. When I say roadmap I certainly don't intend to mean that each step needs to be perfectly detailed.

The field of "AI safety" needs more engineering and less philosophy. But that doesn't require unity around a single roadmap, or any concept of deliverable dates.

Exactly exactly and well said! That is 100% the point. Mind meld complete.

Re:What you wanted to know about AI by maaku on Monday March 16, 2015 @06:13PM (#9214815)
Awesome :)
Re:What you wanted to know about AI by rl3 on Tuesday March 17, 2015 @06:03AM (#9217054)
>The field of "AI safety" needs more engineering and less philosophy.

An alternate phrasing might state that it needs more philosophy and an even greater amount of engineering.

If the field is in its infancy from a technical standpoint, then it probably is from a philosophical one as well.

Re:What you wanted to know about AI by maaku on Tuesday March 17, 2015 @11:35AM (#9218626)
No the philosophical musings have proven to be worse than a distraction IMHO.
Re:What you wanted to know about AI by csallen on Monday March 16, 2015 @02:07PM (#9213153)
I'm not worried about the immediate future at all. We definitely haven't reached the point where hastily-implemented regulations will do more good than harm.

That said, you put it exactly right. The arguments about the potential long-term risk are persuasive regardless of where we are today. It bothers me to see this argument ignored time and time again.

Re:What you wanted to know about AI by maaku on Monday March 16, 2015 @04:10PM (#9214029)
> That said, you put it exactly right. The arguments about the potential long-term risk are persuasive regardless of where we are today. It bothers me to see this argument ignored time and time again.

Actually I am almost completely unpersuaded by the arguments of Bostrom, Yudkowsky, et al, at least in their presented strong form. Superintelligent AI is not a magical black box with infinite compute capacity, the two assumptions which really underlie the scary existential risk scenarios if you look closely. It is not at all clear to me that there will not exist mechanisms by which superintelligences can be contained or their proposed actions properly vetted. I have a number of ideas for this myself.

In the weak form it is true that for most of the history of AI its proponents have more or less ignored the issue of machine morality. There were people in the 90's and earlier who basically advocated for building superintelligent AI and then sitting back and letting it do its thing to enact a positive singularity. That indeed would have been crazy stupid. Kudos to Yudkowsky and others for pointing out that intelligence and morality (mostly?) orthogonal. But it's a huge unjustified leap from "machines don't necessarily share our values" to "the mere existence of a superintelligent amoral machine will inevitably result in the destruction of humanity." (not an actual quote)

Re:What you wanted to know about AI by csallen on Monday March 16, 2015 @04:33PM (#9214198)
You're right: AGI might not be as powerful as some predict it can be. And there might be a way to contain it if it is. And we might be able to enforce said containment strategy on researches worldwide. And the AGI might be benevolent even if it is super powerful and uncontainable.

The authors above are unpersuasive in arguing that these things are inevitable, but they are persuasive in arguing that these things are possible. And if they are possible, then what matters to me is their likelihood. Based on current estimates, are we talking about a 50-50 chance here? 80-20? 99-1? 99.999-0.001?

If we're talking about the future of the human race, these chances matter. But instead it seems everyone is just waving their hands at the what-ifs simply because they aren't 100%.

Re:What you wanted to know about AI by Micaiah_Chang on Monday March 16, 2015 @10:11PM (#9215872)
I agree with a lesser form (timescale on which the AI self improves is probably going to be long enough for humans to deal with it) of your first argument, but I'm confused at how, given 'merely' lots of compute capacity can be countered by 'a number of ideas for this myself'.

Security for human level threats is already very poor, and we already have a relatively good threat model. If you suppose that an AI could have radically different incentives and vectors than than a human, it seems implausible that you could be secure, even in practice. I suppose you could say that these would be implemented in time, but it's not at all clear to me that a humanity which has trouble coordinating to stop global warming or unilateral nuclear disarmament would recognize this in time.

On the other hand, I'm slightly puzzled by why you think there's a huge unjustified leap between lack of value alignment and threat to the human race. Does most of your objection lie in 1) the lack of threat any given superintelligent AI would pose, because they're not going to be that much smarter than humans or 2) the lack of worry that they'll do anything too harmful to humans, because they'll do something relatively harmless, like trade with us or go into outer space?

For 1, I buy that it'd be a lot smarter than humans, because even if it initially starts out as humanlike, it can copy itself and be productive for longer periods of time (imagine what you could do if you didn't have to sleep, or could eat more to avoid sleeping). And we know that any "superintelligent" machine can be at least as efficient as the smartest humans alive. I would still not want to be in a war against a nation full of von Neumanns on weapons development, Muhammads on foreign policy and Napoleons on Military strategy.

For 2... I would need to hear specifics on how their morality would be close enough to ours to be harmless. But judging by your posts cross thread this doesn't seem to be the main point.

By the way, I must thank you for your measured tone and willingness to engage on this issue. You seem to be familiar with some of the arguments, and perhaps just give different weights to them than I do. I've seen many gut level dismissals and I'm very happy to see that you're laying out your reasoning process.

Re:What you wanted to know about AI by eli_gottlieb on Tuesday March 17, 2015 @03:16AM (#9216660)
I just want to point out: the history of software, just regular software, has been typified by the New Jersey approach and the MIT approach. The former consists in just hacking together something that kinda-mostly works, releasing fast, and trying to ameliorate problems later. The latter consists in thoroughly considering what the software needs to do, designing the code correctly the first time with all necessary functionality, (ideally) proving the code correct before releasing it, putting it through a very thorough QA process, and then releasing with a pre-made plan for dealing with any remaining bugs (that you didn't catch in the verification, testing, and other QA stages).

Only the latter is used for programming important things like airplanes, missile silos, and other things that can kill people and cost millions or billions of dollars when they go wrong -- the things in which society knows we cannot afford software errors.

I don't think it's even remotely a leap to say that we ought to apply only this thorough school of engineering to artificial general intelligence, whether it's at the level of an idiotic teenager or whether it's a superintelligent scifi thingy.

Now, if we think we can't really solve the inherent philosophical questions involved in making a "world optimization" AI that cannot go wrong, then we should be thinking about ways to write the software so that it simply doesn't do any "world optimization", but instead just performs a set task on behalf of its human owners, and does nothing else at all.

But either way, I think leaving it up to New Jersey kludging is a Very Bad Idea.

Re:What you wanted to know about AI by maaku on Tuesday March 17, 2015 @11:41AM (#9218672)
No one is making "world optimizations" engines. The concept doesn't even make sense when he wheels hit the road. No AI research done anytime in the foreseeable future would even be at risk of resulting in a runaway world optimizer, and contrary to exaggerated claims being made there would be plenty of clear signs something was amis if it did happen and planty of time to pull the plug.
Re:What you wanted to know about AI by eli_gottlieb on Tuesday March 17, 2015 @04:35PM (#9221242)
I think you missed the last sentence: your software doesn't need to be a "runaway world optimizer" to be a very destructive machine merely because it's a bad machine that was put in an important job. Again: add up the financial and human cost of previous software bugs, and then extrapolate to consider the kind of problems we'll face when we're using deterministically buggy intelligent software instead of stochastically buggy human intellect.

At the very least, we have a clear research imperative to ensure that "AI", whatever we end up using that term to mean, "fails fuzzily" like a human being does: that a small mistake in programming or instructions only causes a small deviation from desired behavior.

Re:What you wanted to know about AI by darkmighty on Monday March 16, 2015 @03:10PM (#9213578)
Maybe it's precisely because the it can't be argued against that it's not much use? There's no way we could tell it will ever happen in the moment, no way to tell how intelligent we can make computers, and so on. You can't reliably argue for or against.

So what's the point? I mean, you may want to form a group and discuss those scenarios, but it's not excusable to spread fear mongering in the media with no substance like that. It's unscientific in a way.

We don't really need to start writing traffic rules for flying cars.

Re:What you wanted to know about AI by csallen on Monday March 16, 2015 @04:15PM (#9214063)
I disagree that it can't be argued against. It's a set of logical steps, any one of which can be critiqued:

1 - There's no real dispute that computers will eventually be as "powerful" as brains. In fact, it's likely that they will one day be far more powerful.

2 - Assuming the brain evolved naturally, there's no reason to assume humans won't eventually duplicate the software once we have the hardware. It's really only a matter of when.

3 - An AGI with equal/more power than the human brain will, eventually, be able change its own code and improve upon itself.

4 - Given the characteristics of intelligence as it has been observed thus far in nature, even small increases in IQ lead to massive improvements in capability that are difficult for lesser intelligences to comprehend. A sufficiently advanced AGI would not only be highly capable, but would quite possibly be too smart for us to predict how it might behave. This is a dangerous combination.

Further complicating things, we might hit step 2 on accident, without realizing that we hit it. Or some group might accomplish step 2 in secret.

What I'd like to see someone do is argue that the chance of these things happening so small and/or the consequences so minuscule that it's not worth worrying about or planning for.

Re:What you wanted to know about AI by maaku on Monday March 16, 2015 @06:57PM (#9215072)
Step 4 is not at all obvious and would require some significant justification.

General intelligence is, well, general. My primate brain may not be able at all to intuit what it is like to move on a 7-dimensional hyper-surface in an 11-dimensional space, but thanks to the wonders of formalized mathematics I can work out and tell you anything you want to know about higher-dimensional geometry. If the super-intelligence itself is computable, and we have the machinery to verify portions of the computation ourselves, it is in principle understandable.

Of course there will be computational limits, but that's hardly anything new. There is no single person who can tell you absolutely everything about how a Boeing 787 works. Not even the engineers that built the thing. They work in groups as a collective intelligence, and use cognitive artifacts (CAD systems, simulators, automated design) to enhance their productive capacity. But still, we manage to build and fly them safely and routinely.

There is no law of nature which says that human beings can't understand something which is smarter than them. Deep Blue is way smarter than me or any other human being at Chess. But its operation and move selection is no mystery to anyone who cares to investigate its inner workings.

Re:What you wanted to know about AI by csallen on Monday March 16, 2015 @10:11PM (#9215871)
I agree that it's not impossible for us to understand something smarter than us. But I don't particularly like your examples. Understanding a static well-organized human-designed system like a 787 or a chess-playing algorithm is far simpler than understanding the thoughts and desires of a dynamic intelligence.

A better analogy would be to stick to IQ. How long would it take a group of people with an average IQ of 70 to understand and predict the workings of the mind of a genius with an IQ of 210? Probably a very long time, if ever. What if the genius was involved in direct competition with the group of people? She'd be incentivized to obscure her intentions and run circles around the group, and she'd likely succeed at both.

Just how intelligent might an AGI become given enough compute power? A 3x IQ increase is a conservative prediction. 10x, 100x, or even 1000x aren't unimaginable. How can pretend to know what such an intelligence might think about or care about?

Re:What you wanted to know about AI by AndrewKemendo on Monday March 16, 2015 @03:51PM (#9213909)
We don't really need to start writing traffic rules for flying cars.

This is a great metaphor and one I wish I would have thought up sooner.

Re:What you wanted to know about AI by maaku on Monday March 16, 2015 @04:19PM (#9214089)
It's a terrible metaphor, or at least it argues against your own position. If flying cars were in fact something on the horizon, we would need discussions about air spaces, commute lanes, and traffic rules. Otherwise lethal mid-air collisions would be much more probable, resulting in significant bystander and property damage as these wreckage fall out of the sky.

And, no surprise, we're seeing exactly this discussion going on right now about drones.

Re:What you wanted to know about AI by im3w1l on Monday March 16, 2015 @02:42PM (#9213387)
>For example, find me someone who can tell me what kind of hardware we need for an AGI.

My guess: Something with tons of flops (or even fixed point ops, if that would be easier). Everything else secondary.

Re:What you wanted to know about AI by jdietrich on Monday March 16, 2015 @10:41PM (#9215966)
I cannot disagree more strongly.

Look at the technologies underpinning the internet. We badly screwed up a lot of things that are very difficult to fix in retrospect, because we didn't invest enough time and effort into foreseeing the consequences.

Nobody worried about the fact that SMTP allows people to send millions of unsolicited messages at practically zero cost until it was far too late to fix the protocol. It is entirely plausible that an RFC describing a proof-of-work scheme could have fixed the problem from the outset, but now we're stuck with Bayesian filters and black/whitelists that sort of work acceptably most of the time.

The issues we're seeing regarding surveillance and censorship could have been hugely ameliorated if our protocols were designed with greater foresight, if people like Cerf and Berners-Lee were more aware of those risks.

We look back in horror at the naive techno-utopianism of the past and the harms that resulted - tetraethyl lead, asbestos, nuclear fission, CFCs. They were all heralded as miracles of the modern age, but caused harms that we're still dealing with today. The technology industry needs to be far more circumspect about the hazards of what we do. We need less hype about changing the world, and more sleepless nights about the people we might be harming.

We need to be having big discussions right now about every aspect of technology, before we open Pandora's box. For example, the World Anti-Doping Agency already have rules prohibiting genetic doping in sport, because they know that it's a potential risk. They would rather figure out their policies now in the cold light of day, rather than in a panic when the Russian team arrive at the 2032 Olympics with a team of superhuman mutants.

AGI is a starting point for broader discussions about how society deals with increasingly powerful and ubiquitous computing technologies. Maybe it will happen, maybe it won't, but I don't think that's particularly important; What we can all agree on is that machine intelligence will only become more disruptive to society.

We need to plan now for what we would do if certain technologies come good. Driverless cars might never happen or they might be five years away, but we need to figure out what to do with the millions of Americans who drive for a living before we put them all out of work. What would be the social and ethical implications of brain-computer interfaces, of algorithmic decision-making in medicine or the criminal justice system, of ubiquitous facial recognition and tracking? How can we plan to mitigate those harms? If we don't give serious thought to those kinds of questions, then we're sleepwalking into the future.

Re:What you wanted to know about AI by bglazer on Monday March 16, 2015 @03:37PM (#9213794)
> For example, find me someone who can tell me what kind of hardware we need for an AGI. Can't do it, because no one has any idea.

Wouldn't the human brain qualify?

Re:What you wanted to know about AI by AndrewKemendo on Monday March 16, 2015 @09:31PM (#9215746)
No, it's not Artificial.
Re:What you wanted to know about AI by themgt on Monday March 16, 2015 @05:56PM (#9214700)
Really I would describe these people as bikeshedding. They're pontificating about the only abstractions they understand, some hand-wavey idea of AI and then the sci-fi short stories of the imagined dangers. Because if you can't do, fanfic.
Re:What you wanted to know about AI by davmre on Tuesday March 17, 2015 @10:47AM (#9218257)
I don't pretend that an argument from authority resolves this debate, but the fact that people like Stuart Russell take these arguments seriously implies that there's a bit more substance there than you're acknowledging.

To actually argue the point a little bit, the theory of expected-utility-maximizing agents is pretty much the framework in which all of mainstream AI research is situated. Yes, most current work is focused on tiny special cases, in limited domains, with a whole lot of tricks, hacks, and one-off implementations required to get decent results. You really do need a lot of specialized knowledge to be a successful researcher in deep learning, computer vision, probabilistic inference, robotics, etc. But almost all of that knowledge is ultimately in the service of trying to implement better and better approximations to optimal decision-theoretic agents. It's not an unreasonable question to ask, "what if this project succeeds?" (not at true optimality -- that's obviously excluded by computational hardness results -- but just at approximations that are as good or better than what the human brain does).

Do Nick Bostrom and Eliezer Yudkowsky understand when you would use a tanh vs rectified linear nonlinearity in a deep network? Do they know the relative merits of extended vs unscented Kalman filters, MCMC vs variational inference, gradient descent vs BFGS? I don't know, but I'd guess largely not. Is it relevant to their arguments? Not really. You can do a lot of interesting and clarifying reasoning about the behavior of agents at the decision-theoretic level of abstraction, without cluttering the argument with details of current techniques that may or may not be relevant to the limitations of whatever we eventually build.

Re:What you wanted to know about AI by JoeAltmaier on Tuesday March 17, 2015 @10:57AM (#9218325)
All that talk about maximizing utility, isn't about intelligence at all. Its about a sensory feedback loop maybe. Intelligence is what lets you say "This isn't working. Maybe I should try a different approach. Maybe I should change the problem. Maybe I should get a different job". Until you're operating at that meta level, you're not talking 'intelligence' at all, just control systems.
Re:What you wanted to know about AI by davmre on Tuesday March 17, 2015 @11:14AM (#9218447)
That's not the definition the mainstream AI community has taken, for what I think are largely good reasons, but you could define intelligence that way if you wanted. It's only a renaming of the debate though - instead of calling the things we're worried about "intelligent machines", you'd now call them "very effective control systems".

The issue is still the same: if a system that doesn't perfectly share your goals is making decisions more effectively than you, it's cold comfort to tell yourself "this is just a control system, it's not really intelligent". As Gary Kasparov can confirm, a system with non-human reasoning patterns is still perfectly capable of beating you.

Re:What you wanted to know about AI by JoeAltmaier on Tuesday March 17, 2015 @12:24PM (#9219038)
Yeah you can imagine a lizard brain being introduced into a biomechanic machine to calculate chess moves. It doesn't make a lizard more intelligent, or even add intelligence to the lizard.

If we don't regard intelligence as something different from control, then I guess birds are the most intelligent because they can navigate complex air currents. Etc. That is a poor definition of intelligence, because its not helpful in distinguishing what we normally mean by 'smart' from mechanistic/logical systems.

And the discussion of rogue AIs is all about intelligence gone awry. Does anybody fear a control system that mis-estimates the corn crop? No, its about a malicious non-empathetic machine entity that coldly calculates how to defeat us. And that requires more that the current AI's are delivering.