Know your advisers and use them. The main problem to solve is knowing when to listen --- and when not to. Look for commonalities in what they're saying. If you average their feedback together, which concerns float to the the top of the stack?

If it's a social news site, why not have them enter each concern as a separate item? To the extent that others agree, that concern will get up-voted. Then there's transparency for all involved when you listen and when you don't. And they end up using your product more.