Re:Is Intelligence Self-Limiting?
by robertskmiles on Sunday March 11, 2012 @08:10AM
(#3690222)
> It's hard to balance the ability to create new, more useful utility functions with prevention of creating a utility function at odds with what the original entity valued.
I think that for such an AI, the concept of a "more useful utility function" would be a nonsense. The AI's definition of 'useful' is the utility function. No other utility function can ever rate higher against the current utility function than the current utility function does.
It's hard to balance the ability to create new, more useful utility functions with prevention of creating a utility function at odds with what the original entity valued.