So that makes me think that maybe they just don't know why it works, so the modified one still works for the same unknown reason. Probably not though. Like they say, they need more tests.
Pretty please, somebody confirm my reading comprehension skills are complete garbage? :)
My next suspects for error would be that the device is propelling itself through heat dissipation or electrostatic ally by throwing off charged particles. Both of those possible confounding factors are testable, either by measuring heat via IR imaging and computing what propellant effect that heat would have of by dummying up the null device to produce and dissipate an identical amount of energy to the non-null device.
But honestly, I'm a hacker. I'd go rent me a vacuum chamber and put it on a whirlygig and see if it makes it spin. If it spins faster and faster and faster until it flies apart (well okay maybe I'd turn it off before loss of structural integrity), something is definitely off in our understanding of physics and God really will let us have hoverboards. So give me a hoverboard.
Most of the time bias creeps in at the analysis stage. In my experience. Deciding what data to keep and what to reject, deciding which fitting algorithm is working correctly, etc.
If we find out that inanimate objects behave differently if the operator does/doesn't believe in their success (as patients do), then that'd be much bigger news than engine development.
http://ntrs.nasa.gov/search.jsp?R=20140006052
> Thrust was observed on both test articles, even though one of the test articles was designed with the expectation that it would not produce thrust. Specifically, one test article contained internal physical modifications that were designed to produce thrust, while the other did not (with the latter being referred to as the "null" test article).