Coincidentally, this just came up: http://news.ycombinator.com/item?id=1199230 It is almost exactly what I mean. Unfortunately, if the results are sufficiently bad there's actually no salvaging the science; meta-analysis can improve things but still requires good data to start with.