News for hackers. Stuff that matters.
Re:What Darwin's Theory of Evolution Reveals About Artificia... by endtime on Friday June 22, 2012 @07:08PM (#4149354)
> "optimize [x], but without doing anything obviously stupid that I wouldn't want you to do"

This is similar to the basic idea of Yudkowsky et al's Coherent Extrapolated Volition, the gist of which is that an FAI's metaethical function should be some aggregated of what it believes people would want it to do if they had the knowledge and intelligence(/cycles) that the AI has. (I believe there exist updates/improvements to CEV, but I don't know much about them.)

There are no comments on this comment yet.