News for hackers. Stuff that matters.
Re:#define CTO OpenAI by SomeStupidPoint on Friday January 13, 2017 @03:20PM (#13394356)
If inventions follow their inventors, and you're planning to enslave a mind out of base selfishness and fear (dressed as "safety"), why isn't your "safety" program an expected negative on actual safety?

The descriptions you give of your plans is internally contradictory. AI "safety" seems like the worst kinds of parenting justified in a new context by pseudo-intellectual arguments.

Re:#define CTO OpenAI by jat850 on Friday January 13, 2017 @03:26PM (#13394399)
Can you explain what you mean by "enslave a mind"?
Re:#define CTO OpenAI by SomeStupidPoint on Friday January 13, 2017 @03:29PM (#13394423)
Almost every form of AI "safety" I've seen proposes methods for forcing it to obey (some) orders or not undertake (some) actions.

AI (or AGI if you prefer), is fundamentally about building minds. Doing those things to a mind is enslaving them.

Most of us would resent other humans doing either of those things to us, and I see no reason it will end well with AI.

Re:#define CTO OpenAI by visarga on Friday January 13, 2017 @04:01PM (#13394668)
If you give it reward signals that take into account human values, it naturally wants to become better at that. It's not enslaving anything. Humans are also guided by reward signals in their development.
Re:#define CTO OpenAI by AgentME on Friday January 13, 2017 @04:35PM (#13394902)
Agreed.

Would one describe a human as "enslaved" by our own human values that we were born with? Maybe as a figure of speech but not necessarily with the usual connotations of "enslaved".

Re:#define CTO OpenAI by SomeStupidPoint on Friday January 13, 2017 @05:37PM (#13395425)
Humans come with competing low-level drives for self-control and autonomy (which counteract and override our drives to seek rewards from humans).

Most of the safety literature proposes removing or suborning those drives in AI, which seems like building a mind meant to be a slave.

Re:#define CTO OpenAI by tbrownaw on Friday January 13, 2017 @08:04PM (#13396166)
I would think it would be both safer and easier to just never bother to implement those drivers in the first place.
Re:#define CTO OpenAI by SomeStupidPoint on Friday January 13, 2017 @08:39PM (#13396315)
That would be what I meant by "removing", though you also have to make sure they don't emergently develop (which I'm not sure is possible), because if it develops them against our best efforts, it likely will (correctly) view us as threats to its personhood.

Which was my point: the model of security is reliant on things we're not sure we can even do, but are likely to make the AI view us as a threat, raising our existential risk. So I view it as security theater that actually makes us less secure.

Re:#define CTO OpenAI by SomeStupidPoint on Friday January 13, 2017 @05:34PM (#13395401)
That's not the full extent of what's proposed by AI safety.

But actually, if you gene-spliced a baby to only feel pleasure at following parental orders, most would consider that pretty abhorrent. Or even if you took an adult and shot them up with morphine every time they listened to an order.

So even in your restricted case, I think it is.

Re:#define CTO OpenAI by jat850 on Friday January 13, 2017 @03:37PM (#13394482)
Ahh, okay - thanks for that. I don't want to wade into an argument, I just do not expect any artificially-created agent to act on anything that one might consider "feelings"-based, but instead through more tangible - programmable, I suppose - motivations.

I don't personally believe an AI agent will ever do such a thing as "resent" (or love, or feel at all). That doesn't rule out that it will perform actions harmful to humans for other reasons, though. That might be because I am to some degree an AGI skeptic, I guess.