His advantage in 538 is that he is using very sound statistical methods to combine multiple polls to increase the effective sample size and shrink confidence intervals. The complexities there are all related to increasing the individual pollster's historic performance and simulating the actual election.
His model has a bunch of other stuff too, though: estimates of how current jobs numbers effect the election, "state fundamentals" based on demographic variables fit to historical election data, etc. I'm not sure these things are adding much predictive power, especially given that they're generally fit to sparse, heterogeneous, and noisy data sources (e.g. the 11 Presidential elections 1968-2008).
On the other hand, one difficulty in evaluating it is that a good portion of the work his model is doing is not aimed at giving the point estimate, but the distribution and probabilities, to give an accurate probabilistic picture of the current state of the election at any given point, integrating various available data and historical trends. The goal, which is sensible, is to take a more rigorous approach to numbers you hear thrown around like, "when an incumbent is up by x% y weeks before the election, they win z% of the time". But even now in retrospect we can't easily evaluate how good those distributions were: we can see how well his point estimate fared, but with only one instance of how the election turned out, can't evaluate his overall picture's accuracy.
I think he adds tremendous value in his write-ups. He explains what he's doing in pretty good detail, and provides some insight into what he believes it means. If you're seriously into it, you'll find places you disagree with him (same as it works in any other field when people are into it), but even if you're just casually interested you'll come away with some additional understanding. Or at the very least some additional questions of your own.
And he manages to write it in such a way that general audiences can pick up some of it, and come away with the impression that there's a lot more there. Not saying it's not a true impression, just that the fact that they get the impression at all is what matters.
How does this work? I was under the impression that to shrink the confidence interval by half you need 4x as many sample points, but there must be more sophisticated statistical methods at work here.
I haven't looked into Nate Silver's numbers, and certainly there would still be a reason for supporting him even if Romney had won tonight, but I think what most people are missing is that it's relatively straightforward to get pretty close to accurate, but incredibly difficult if not impossible to be completely accurate.
The predictions, as they get better, are going to asymptotically approach the actual results.
Yet at the end they decided not to implement it http://www.forbes.com/sites/ryanholiday/2012/04/16/what-the-...
Do you know what caused a team to win?
They Realized (1) people rate movies higher on specific days ( why would you want to implement that into your estimator? And (2) they realized that all the movies that Netflix was asking about we're movies the user was willing to rate, and because users tended to rate movies they like, this biased the results, so basically, they didn't implement the hacking the test
But seriously, congratulations to Nate Silver. I think this vindicates his method, at least for America. He was badly wrong with the last British election, whether because of bad polling data or an insufficient understanding of the system, so I was apprehensive going into this one. But it seems like, for America at least, he's cracked it. I think it's inevitable considering the absurd quantity of polling data available that someone would eventually.
Mr. Silver is one of the most clever people we have on the public stage, but that doesn't mean he has a silver bullet for everything. Rather, it enables him to work in a field that he knows very well and knock it out of the park.
Silver predicted: Conservative 312, Labour 204, Libdem 103 Actual result : Conservative 307, Labour 258, Libdem 57
Either way, it shows that statistical averages of what people say they'll do isn't always a good indicator of what they'll actually do when push comes to shove.
Disclaimer: as a (now rather embarrassed) Lib Dem, I'd quite like these predictions to be more accurate so I don't get my hopes up again.
His US model makes a lot of allowances for things like how self-proclaimed independents are likely to vote.
It didn't work as well in the UK though.
how did he get it so wrong?
In any case, we don't know if he got it "wrong" unless we know what % chance he gave this outcome. (He tries to account for the likelihood that polls are misleadingly biased, and that would correlate across multiple seats.)
politics is not nearly as important. :-)
http://www.colleyrankings.com/election2012/
Their predictions were basically the same as Sliver's, missing only on Florida (they manually broke a tie in the model on Virginia so count that however you'd like).
It was 48/50 in 2008 and 49/50 in 2004.
His baseball projection system, PECOTA, was extremely complex, but barely, barely outperformed a simple 3-year weighted average with an age component (Marcel), and some years was worse. Other projection systems that didn't take player comps into account were better overall than PECOTA (CHONE being the best, before the creator was hired away by a team).