There was a good documentary on polling on BBC Radio 4 in light of their very weak performance in IndyRef and the UK General Election. One pollster said that he actually has a high regard for outliers which statistical aggregations do not capture. In cases where the majority of pollsters got it horribly wrong, there have been outliers at some significant distance from the herd that have been on the money but ridiculed at the time.

The reasoning is that pollsters have herd behaviours - they believe publishing an outlier they will lose them credibility so they suppress inconvenient results and will rework methodologies and weightings to avoid it. For many, election polling is a loss leader - its publicity to get their name out there for commercial services. The credibility of the outlier is increased because it has escaped the herd pressure despite the incentives.

I think problem is deeper. If one predicts heads as result of coin toss and other predicts tails, than one of them will be right. It says nothing about their abilities to predict. We always can find some outlier who predicts results different from majority of pollsters. The keyword is 'always': if such a possibility always exists (probability of existence P=1), then we can't extract any information from it (-log_2 P == 0).

To extract information we need to carefully revise methodology behind outlier predictions, and such an activity is a complex task for skilled data analyst.