News for hackers. Stuff that matters.
Re:Introducing OpenAI by JoshTriplett on Friday December 11, 2015 @05:01PM (#10720419)
The idea is not that we should build and (try to) suppress a sentient AI; that would be a bad idea for numerous reasons. However, we don't necessarily need to build a sentient AI in the first place; we can build a process that has reasoning capabilities far above human without actually having agency of its own.
Re:Introducing OpenAI by TelmoMenezes on Friday December 11, 2015 @05:23PM (#10720539)
> we don't necessarily need to build a sentient AI

How do you know if an AI is sentient or not? We don't even know what sentience is. For all we know, maybe the computation of a Mandelbrot set is conscious.

> capabilities far above human without actually having agency of its own

What does this mean? Doesn't a chess-playing algorithm have agency of its own? What about a self-driving car? People say: "oh, but ultimately they are only doing what they were programmed to do". Sure, but so are we. It's just that our programming is done in a much less straightforward way.

Consciousness and free-will are open problems. Theories about these things are not even wrong, because people can't seem to agree on a definition. Personally, I suspect that "free will" is meaningless* and that consciousness is qualitatively beyond our current level of understanding of realty.

* I suspect that this meaningless concept was introduced to solve the conundrum: "if god is good why does he allow for bad things to happen?". And the answer they came up with was "so that we can have free will". But think about it, what does that even mean?

Re:Introducing OpenAI by baconner on Friday December 11, 2015 @05:09PM (#10720450)
See I think that's exactly where it becomes complicated. Can an entity with reasoning capabilities far beyond that of humans have its agency suppressed successfully? And is it ethical to do so or is that internally designed suppression somehow ethically different from the external suppression applied against human slaves?

If you could engineer a human being with his/her agency removed so that you could use their reasoning skill without all that pesky self will would that be ethical?

Re:Introducing OpenAI by JoshTriplett on Friday December 11, 2015 @06:36PM (#10720917)
The way you're asking the question implies that reasoning inherently has agency/sentience that needs suppressing. It doesn't need to; there's nothing to "suppress".
Re:Introducing OpenAI by baconner on Friday December 11, 2015 @08:59PM (#10721446)
We don't know that one way or another since such a machine doesn't yet exist. I'm suggesting that perhaps high level reasoning and sentience go hand in hand although I can't say that with any certainty.