Oh god no. A thousand times no.
Humans are masters at hiding sociopathy behind cultural and political obfuscation.
A machine that looks at human leaders and generalises human values from them is absolutely guaranteed to be a monster.
>I'd be a lot more comfortable if metaethics and related philosophy advanced to the point where we didn't have to rely on indirect normativity.
We already have any number of perfectly sustainable ethical systems. What we don't have is the ability to follow them in practice - partly because all political systems tend to reward power and influence with more power and influence, and partly because even when there are token checks and balances (e.g. in the form of representative democracy) it's so easy to game them with appeals to irrationality and emotion.
This rarely works out well for most of the population.
But I don't much like the content of the proposal. I think you can have extremely useful intelligence amplification without any need for agency or personality.
Much better modelling along the lines of 'If you do this, this will be the outcome' would be a game changer on its own.
Humanity has already solved ethics, so there's no reason for us to give up and ask AI to define ethics for us.
That's the point...
>Humanity has already solved ethics
Where on earth do you get this idea?
Here's a relevant essay http://medium.com/@yanazendo/how-to-be-lazy-and-successful-a...
I'd be a lot more comfortable if metaethics and related philosophy advanced to the point where we didn't have to rely on indirect normativity. But I don't think this is something you can just throw money and people at, because this is a field which is sufficiently difficult and hard to judge, that work which isn't first-rate tends to detract from progress by adding noise rather than help.