As the OPERA result showed, it has the problem that if you don't understand everything in your experiment perfectly (which is difficult to do in a very large, complicated experiment) you run the risk of embarrassing yourself by making some obvious-in-retrospect mistake and publishing an obviously absurd result. But in the long run it's not so bad a price to pay to avoid the sort of confirmation bias that Feynman was talking about.
[1] http://people.psych.cornell.edu/~jec7/pcd%20pubs/simmonsetal...
How does that help when the prediction matches the measurement but the experiment is flawed ?
In a nutshell, we know that science more or less progresses from one theory to another after the existing theory has been falsified (Popper), but this transition does not happen overnight, even after a falsifying result has been put forth. That is because science is really structured with a "hard core" hypothesis (say, in physics, that nothing moves faster than c), and various "auxiliary hypotheses" that bear the brunt of criticism before the core belief is attacked (say those that posit the integrity of the measurement instrument, for example). And that is all that is going on here. There are many, many theories that are going to be dismantled before physics transitions to something post-relativity. It's just the way the assumptions are structured.
While this may seem trivial, we see this playing out in the battle of classical against behavioral economics. Are the results of experimental psychology admissible in the criticism of certain economic models? Models don't claim to be perfect, after all...
1. http://en.wikipedia.org/wiki/Imre_Lakatos#Research_programme...
And don't forget there is a check and balance in the form of meta-studies and statistics that are sometimes uncanny can exposing bias and flaws.
Finally - this highlights the dual nature of 'experimental' vs 'theoretical' physics, and they work so well in tandem.
When Millikan's experiment was new it was new; people hadn't done it before, there was slop to work out, and the theory was miles behind. These things take time to converge on a consensus. Sometimes a long time. Also, money.
How would such dynamics pan out for experiments that aren't amenable to the sort of polite gradual adjustment as Millikan's was?
The real lesson is that this sort of thing (fudging numbers, lack of scientific integrity) is quite prevalent but most of the time goes unnoticed because there isn't something as dramatic as with the Millikan experiment to show it plainly.
1. Hubble was actually measuring the local motion of nearby galaxies, the so called Local Group. These galaxies are too close for Hubble expansion to matter, they're just gravitating.
2. The actual value for the Hubble constant he measured was way off, by something like 10x.
1. Hubble's draftsman made an error while preparing his famous diagram. From Figure 1 of his paper (http://adsabs.harvard.edu/abs/1931ApJ....74...43H), you'll notice that the vertical axis is in "km" rather than "km/sec."
2. Measurements of the Hubble Constant over time: http://www.pnas.org/content/101/1/8/F2.large.jpg For some reason it's not plotted logarithmically so it's hard to see the convergence towards ~70 km/s/Mpc in the last twenty years.
Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It's a little bit off because he had the incorrect value for the viscosity of air. It's interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan's, and the next one's a little bit bigger than that, and the next one's a little bit bigger than that, until finally they settle down to a number which is higher.
Why didn't they discover the new number was higher right away? It's a thing that scientists are ashamed of - this history - because it's apparent that people did things like this: When they got a number that was too high above Millikan's, they thought something must be wrong - and they would look for and find a reason why something might be wrong. When they got a number close to Millikan's value they didn't look so hard. And so they eliminated the numbers that were too far off, and did other things like that...