At Google (while Amit was there) trained manual raters were very important part of determining whether a change should be launched or not (especially for relevance). I'm not sure how the process changed since he left, as I see much more wrong auto-corrections by Google.