Correlation does not equal causation. Another possible explanation: She was a monster at predicting project success and worked her way onto projects that were going to succeed with or without her.
Sure, but if causation is effectively impossible to rigorously determine, it can end up being all you've got. If you've got the choice between going with someone whose presence correlates with success on the project and one who does not, my inability to be rigorously sure about causation isn't going to make me lose much sleep at night when I chose the correlative one.
I'm coming to dislike the citing of "correlation does not equal causation" when there's no way to determine causation at all, and when scientific certainty isn't the question at hand. At that point it's an excessively-powerful criticism, one that can't be discharged, so is it really a useful criticism at all if so?
This is perfectly understandable, however this particular discussion is one where the difference between correlation and causation is appropriate. We are talking specifically about paying programmers by their "productivity." If you want to say that "productivity" is defined as the correlation between a person and project success regardless of whether there is a causal relationship or not, and regardless of whether they engage in programming activities, project management activities, picking good project activities, discussion activities, or even just making everyone else espresso so they can produce working code, that's fine.
But what we're saying in that case is that we can't measure the productivity of a programmer, we can't establish a relationship between programming activities on the scale of a single person. I agree that the correlations you can observe are perfectly useful for management and that one can deliver great (or working, or valuable) software without an objective metric of programmer productivity. I agree that this elusive metric may not be necessary. It may not even be useful, as I tried to demonstrate elsewhere when I discussed Ned, Fred, Ed, and Jed.
But that really underscores my point: We can't tie compensation to programmer productivity because we can't measure it. Your point seems to be that we don't have to tie compensation to programmer productivity, that we can tie it to correlation with project success, for example.
Fine with me, I'd say we're in violent agreement and that our stances are compatible.
OTOH, having a project success divining rod could have its own value.
Even if she was just a monster at predicting project success, I'd still want her on my team. Can you imagine how useful it would be to have someone able to consistently predict project success working with you?
(Interestingly, there's a less flattering interpretation of the data as well. Given a large enough organization that promiscuously shuffles people onto new projects, some of which randomly succeed, someone is going to have randomly ended up on all the successful projects. We'd then look at them and say "Look how awesome they are!", when really they just got lucky over and over again and we're looking because they got lucky. This requires that the organization size be large relative to the number of possible combinations of projects, though, which becomes increasingly unlikely over time. It's like the people who look at Warren Buffett and proclaim "he just got lucky for 40 years in a row", then do out the math and realize that the chance of someone being that consistently lucky is several million to one.)
Fair point, but what would you pay for knowing in advance what projects are going to succeed and by inference which may fail...
Correlation not implying causation is a big deal because it's possible to draw (probably exponentially) many alternative causal chains than the one that you're discovering correlation along.
If the above isn't the case, and it's at least theoretically possible to design experiments like that, then correlation does[1] equal causation.
[1] Sort of. See pretty much anything written by Judea Pearl.
She was the Shane Battier of software.
(I happen to be reading a book on basketball right now.)
I recall reading years ago (in Peopleware or one of its contemporaries) about a company's evaluation of one of its coders. She was definitely mediocre by every measure they had. But someone noticed that every project she was on succeeded, over many projects and many years. Though she wasn't a monster at the keyboard, something she brought to the team engendered success. How productive was she? Would you want to hire her and have her on your team?
You must measure what you actually care about. Measuring things that you think are factors is fine and noble, but if you're not measuring the actual "product" of "productivity" then you'll never know how well your factors correlate with the real goal.