I wonder if this was part of some rare coincidence where their inputs were perfectly synchronized but out of phase in such a way that it made it difficult to consider the obvious. If you pull back but feel the plane and/or stick move laterally, and especially across multiple inputs without a clear pattern, it might more easily occur to you to think the person sitting next to you is also providing inputs. Whereas if you're synchronized across many inputs, that's just something that doesn't typically happen in nature, which tends to always have some stochastic quality; instinctively it's something you might assume would only involve a machine.