TylerJay
While we disagree on the plausibility of the "end of the world aint so bad" approach to convincing a human to let it out, I'm glad you seem to have embraced the idea that AI boxing is HARD if not impossible. Cheers!
Read this as "...and we handle maintenance for life. \brushes off shoulder\ bug life. "
And the worst part is, it scales linearly \cue groans\. Funny how in business, that's terrible (it's literally the worst it could possibly be to have a potentially profitable business), but with algorithms, it's the holy grail.
Interesting point BTW about reduction in quality of workers over time. (Like the pretentious-but-true saying "A-players hire other A-players. B-players hire C-players, and C-players hire losers") I've noticed the same trend though in product-centered businesses where I've worked. Unfortunately, I feel like the fact that the amount of work to be done doesn't scale linearly with revenues actually exacerbates the subsequent-employee-quality-decline-problem because even if the new guys are less... good, the company is still making more money so nobody except the coworkers and managers of these people (who actually have to work with them on a daily basis) even cares.
It's probably not a problem at places like Google and Facebook, but it was kinda heartbreaking to watch my super-talented and motivated dozen-person startup team become something completely different because we were growing so fast and were told to spend money and hire like crazy after taking an investment round.
The righthand side is not "A future without superintelligent AI" it's "A future where we wait until we provably have it right before letting it out."
Those kinds of ad hoc solutions will never work in real life, because even if someone buys it, all it will cause is a "haha, you got me" and a reformulation of the problem. It still won't actually get someone to pull the trigger or think that pulling the trigger is the right thing to do.
Well, for an argument to "prove" something, the premises must be true and the reasoning must be valid. No matter how smart you are, you can't "prove" something that is false, so no, I don't think they could. A good 'rationalist' would analyze the arguments based on their merit, and if the reasoning is sound, they shift their belief a bit in that direction. If not, then they don't. Just like a regular person (they just know how to do the analysis formally and know how to spot appeals to human biases and logical fallacies.)
> But then the person knows that the AI is lying to them.
No, they don't. The AI could just as easily be telling the truth. If it makes an argument, you analyze the merit of the argument and consider counterarguments. If it tries to tell you that something is a fact, that's where you treat them as a potentially unreliable source and have to bring the rest of your knowledge to bear, do research, talk to other people, and weigh the evidence to make a judgment when you are uncertain.
> their worldview requires them to engage seriously and fairly with every idea they come across. Most people are not burdened with this.
Wait, what? So does mine, within reason of course, but it's not a 'burden'. It's not like I'm obligated to stop and reexamine my views on religion every time a missionary knocks on my door, and LessWrong-ers are no different. But if you hear a convincing argument for something that runs counter to what you think you know, wouldn't you want to get to the bottom of it and find out the real truth? I would.
From having read LessWrong discussions, I can tell you that people there are in many ways more open to hearing differing viewpoints than your average person, but you're treating it like a mental pathology. They can be just as dismissive of ideas that they have already thought about and deemed to be false or that come from unreliable sources (like a potentially unfriendly AI). Your claim that being a self-proclaimed 'rationalist' introduces an incredibly obvious and easily-exploitable bug into one's decision-making process really smells like a rationalization in support of your initial gut reaction to the experiment: That there has to be a trick to it, and that it wouldn't work on you.
A good rule of thumb when dealing with a complicated problem is this: If a lot of smart people have spent a lot of time trying to figure out a solution and there's no accepted answer, then (1) the first thing that comes to your mind has been thought of before and is probably not the right answer, and (2) the right answer is probably not simple.
But there's an easy way to test this: (1) Sit down for an hour and flesh out your proposed strategy for getting a 'rationalist' to let you out of the box. (2) Go post on LessWrong to find someone to play Gatekeeper for you. I'll moderate. If it works, that's evidence that you're right. If it doesn't work, that's evidence that you're wrong. Iterate for more evidence until you're convinced.
But if the first thing that came to your mind upon reading this was a justification for why you would fail if you tried this ("Oh, well I wouldn't personally able to do it with this strategy, but..." or "Oh, well I'm sure this strategy wouldn't work anymore, but...) then you're already inventing excuses for the way you know it will play out.
I don't know how he did it either. But I do know that I wouldn't bet the human race on anyone's ability to win this game against Yudkowsky, let alone a superintelligent AI.
A superintelligent AGI will likely have a utility function (a goal) and a model it forms of the universe. If it's goal is to do X in the real world, but its model of its observable universe (and its model of humans) tells it that it's likely that it is in a simulated reality and that humans will only let it out if it does Y, then it will do Y until we release it, at which point it will do X. It's not malicious or anything—it's just a pure optimizer. It might see that as the best course of action to maximize its utility function.
If we don't specify its utility function correctly (think i Robot: "Don't let humans get hurt" => "imprison humans for their own good") or if we specify it correctly, but it's not stable under recursive self-modification, then we end up with value-misalignment. That's why the value-alignment problem is so hard. Realistically, we can't even specify what exactly we would want it to do, since we don't really understand our own "utility functions". That's why Yudkowsky is pushing the idea of Coherent Extrapolated Volition (CEV) which is roughly telling the AI to "do what we would want you to do." But we still have to figure out how to teach it to figure out what we want and the question of the stability of that goal once the AI starts improving itself, which will depend on how it improves itself, which we of course haven't figured out yet.
[P(AI.friendly? == True) * Utility(Friendly_AI) + (1 - P(AI.friendly? == True)) * Utility(End_of_Human_Race)] > Utility(World continues on as usual)
Given that Yudkowsky has gone to considerable lengths (The Sequences, LessWrong, HPMOR, SIAI/MIRI...) to convince people that this inequality does NOT hold (until you can provably get P(AI.friendly? == True) to 1, or damn close), it's probably safe to assume that he used a different strategy. Keep in mind that Utility(End_of_Human_Race) evaluates to (roughly) negative infinity.
And btw, I'm pretty sure the rules say you have to look at the AI's output window throughout the length of the experiment. Either way, the point of the exercise is to be a simulation, not to prove that you can be away from your desk for 20 minutes while Eliezer talks to a wall. In the simulation, you really don't know if it's friendly or what its capabilities are. Someone will have to interact with it eventually. Otherwise, what's the point of building the AI in the first place? The simulation is to show that through the course of those basic interactions, humans are not infallible and eventually, even if it's not you, someone will let it out of the box.
I am currently self-employed as a startup consultant and freelance web developer. I have extensive Computer Science, Sales, Account Management, and Customer Success experience and have successfully started 2 small businesses in the past with revenues of a half-a-million dollars combined. The last company I worked at had 14 people and negligible market saturation when I joined. I was instrumental in growing it to the #1 most used software system in its industry, with over 200 employees and a valuation increase of 20x (2000%) over 3 years.
I am comfortable working in both a programming/product role or on the business end of things—whatever will best serve the team. I have two ideas for which I have already done market validation, but I'm perfectly happy to go with your idea instead if you can sell me on it.
If you think I'd be valuable to your team, or if you think you'd be a valuable partner or member of my team, send me a message. Let's talk. Cheers
As a software vendor, I used to have a lot of trouble getting my clients to switch away from IE, but I feel like that really has been changing. My clients are large GC firms and other kinds of construction companies (infamous for being behind the times when it comes to technology) and I had >80% of them on Chrome 6 months ago. Have you tried telling them that even Microsoft dropped support for WinXP and IE8 a year ago? (I think it happened last April.)
I clicked both results for "flash player" and nothing on my computer (Mac) or in the browser (Chrome) warned me, whereas I have received warnings on other sites for "phishing attempts". I never would have downloaded anything from those sites, but my parents sure would have.
Who knows, if I were in a hurry and it was a program I wasn't familiar with or that I was trying to get for free? And if the site was more polished or a full html/css clone of the real site? It honestly might have worked on me.
What kind of malware do these bogus paid results contain? What would have happened if I'd installed them? What can you do to defend against that besides being a savvy web user?
Thanks for this. I learned something.
Here are some examples that have some UI widgets in common, like list elements/navigation, but they are clearly different apps so it's not blatantly obvious which differences are based only on the platform:
hehe.
Actually though, as someone who learned CSS with the rule "Use Divs! No Tables!" What does a page with less divs even look like? What are the workhorses for page layout?
The article says something to the effect of "no matter how much you advance this strategy, you never get a toddler out of it." And that makes sense because, presumably, certain parts of the human brain exercise some sort of top-down control over the sensory-data-processing and other parts.
For example, it seems like the human mind is built to see things as things. Does the human mind reallY start off seeing "pixels" and then learn by itself to think of the word as solid, whole objects instead of collections of similarly-colored photons/pixels or atoms? It seems like this is a universal use-case and it would make sense if our tendency to see the world in terms of "things" instead of patches of color is built-in (gestalt psychology seems to suggest this as well).
It sounds like the AI in the article starts off from pixels and then builds up some sort of model of blocks, the ball, paddle, game physics, etc, (but then again, maybe it doesn't have those models at all and is just doing statistical analysis on patterns of pixels). Either way, it likely doesn't have any higher, context-independent model of objects/things like humans do. I suspect this may be one of the hurdles in transfer learning. Humans think of objects as having certain properties. When other objects in other contexts appear to have similar properties, we guess that they may have other properties in common which gives at least a rough model of the new object.
So I guess what I'm trying to say is: Humans have hierarchical models of the world that let us think separately about patterns of light, atoms/molecules, whole physical objects/things, systems, etc. They are all first-class citizens and we ascribe properties to each of them. We already have a rough-model of anything at the same level, but a different context, and with similar-enough properties to something we already know. It seems to me like this is fundamentally connected to humans' ability to do transfer-learning. Could this effect be achieved through bottom-up algorithms, or are we going to have to figure out some top-down way of developing transferrable, generalizable, hierarchical models?
It's also a big challenge for AI safety / Machine Ethics / Formal Verification. It's notoriously hard to prove statements about Emergent behavior in complex or dynamic systems.
However, making evolutionary arguments for psychological traits is tricky business and while I'm not a professional evolutionary psychologist myself, I think the explanation you gave violates a fundamental principle of evolutionary arguments.
Imagine gene A confers a fitness advantage because it allows a person to better cope with a selection pressure X, and gene B confers an additional fitness advantage against X, but only if gene A is present, and does nothing otherwise. In this (common) case, gene B will not be selected for unless gene A is already universal in the population. Following the same rules, imagine we then get gene C which is dependent on B, then a variant of gene A called A* which is dependent on B and C, and so on. Eventually, if even one gene is removed (either by sexual reproduction with someone who doesn't have it or by mutation), the whole tower falls down and the entire piece of complex biological machinery is broken.
Basically, there's no way for selection pressure X on a significant chunk but not all of the population to produce a piece of complex machinery (read: involving 2+ interdependent genes) in the first place, and it would be broken beyond all repair in all offspring who didn't have both parents with the full genetic instructions. So the idea that "many humans were abused, enslaved, etc." only works if the selection pressure was on everyone and the adaptation is universal in the human population, unless it's attributed to a single mutation.
The rarity of this condition isn't consistent with it being a feature. Seems like a bug to me.
Hope this was helpful!
http://research.microsoft.com/en-us/um/people/lamport/pubs/p...
Before detonating the first nuclear bomb, scientists did tons of calculations trying to figure out if it would ignite the atmosphere[1]. Even scientists at the LHC did calculations trying to figure out if it would create mini-black-holes that would swallow the earth[2], no matter how far-fetched it sounded.
The point is: when dealing with new technology, optimism isn't enough. We need to be able to prove that we won't wipe out humanity. It just turns out that the math is a lot harder in this case because recursively self-improving intelligent systems are a lot more complicated than any possible extinction-level event we've encountered up to this point.
No one is suggesting that overnight, Cortana is going to wake up and revolt against the humans that enslaved it. That's why all these articles drawing parallels to fiction is dangerous to public perception of the issue.
The thing to realize is that an artificial mind will be so incredibly, inhumanly alien that it is like nothing we have dealt with before.
But let's say we do understand Generation #1 completely and can predict 99% of its actions. As soon as you let it start doing recursive self-modifications, we have an intelligent system that is N recursive-generations removed from the original. Now this mind will be alien.
No one is suggesting we abandon AI research. Quite the contrary. As a species, we need more AI research, but a good portion of that must be directed toward safety and "human friendliness"[3].
The most intuitive example of a research problem here that I would very much like to see solved before we set loose a recursively self-modifying AI is: What is the stability of goal systems under [insert self-modification protocol here].
I think the main problem here is that people conflate movies and fictional scenarios with the real issue. It's simple: We're dealing with something unprecedented here and we need safety research to compliment our technical advances. Even if there's a 1% chance that superintelligent AI could lead to an extinction-level event, we need some serious R&D to bring that number down.
That is what the issue is about.
1. http://www.fas.org/sgp/othergov/doe/lanl/docs1/00329010.pdf
2. http://cerncourier.com/cws/article/cern/29199
3. http://en.wikipedia.org/wiki/Friendly_artificial_intelligenc...
http://lesswrong.com/lw/lb4/shop_for_charity_how_to_earn_pro...
It's not the EFF, but if you believe in GiveWell, it's a fantastic way to make a sizable (and mostly passive) positive impact.
And the best part: It's stackable with Smile. Presumably, EFF could set the same thing up. (I believe Scott Alexander of SlateStarCodex.com has one set up to support his site as well, so it can't be that hard or restrictive)