In one sense, this doesn't surprise me. I remember doing lots of ill-advised refactorings when I was younger. Refactoring is a nifty idea, and so it's easy to be enamored of it and eager to apply it like it's a new toy. I'm sure lots of people act this way with new power tools and often the resulting "oops" gets thrown in the scrap bin.
The problem with refactoring code, is that recognizing the "oops" is harder. Probably, the thing still runs and all the tests still pass. It requires putting yourself in the shoes of someone who hasn't seen the thing before. It's not like seeing that you gouged a sanded surface. It's more like realizing that the technique you were so fond of conflicts with the composition. Not only is it subtle, you often also have to be mature enough to swallow your pride.
That said, a properly done poor refactoring is still superior to copy & paste. Why? Because, done correctly, you can pretty much guarantee finding all the places you need to change. With cut & paste, in a large codebase, you need to do some inspired searching and second-guessing -- and this is where the major risk comes in when you have to change something. You're still going to need some of that in a large codebase, even when it's very DRY. The smart move is to minimize that as much as possible, because again, that is where the risk comes in.
Ultimately, refactoring and DRY should be thought of, at its core, as a clerical tool. You need to take steps to ensure that ideas don't get lost in the codebase -- those give rise to bugs. However, you should only take those steps when the downside risk of ideas getting lost definitely outweighs the risk of making a mistake and introducing bugs or decreasing code clarity and flexibility.
I'm disappointed that the key point of my entire article, which is the difference between accidental sameness and essential sameness, was apparently misconstrued as an attack on all refactoring.
A significant number of bugs I've had to fix over the years were the result of just two repetitions. Invariably someone updates one of them and not the other one. Or copies the first one, then updates, then forgets.
This happens over and over and over and over. I'm so sick of it that I want to punch my coworkers (I don't usually get angry quickly but this has been wearing on me for a few years now). So I will continue to refactor at the second copy.
But how could studies on this type of long-term maintainability draw meaningful conclusions? It just seems incredibly intractable from a scientific perspective—a result of experience and thoughtful consideration outweighing formal, documented and trainable techniques.
Big time. The refactoring in this case was ill advised. When things started getting hairy, it should've been backed out.
Piling too much flexibility in one function is a common mistake. A justification for copy/paste it does not make.
I worked at a shop with this rule: don't try DRY until you've seen at least three repetitions. I think this saves one from premature refactoring.
Another way to put it: Refactor when the code speaks to you, that is when need is evident. Keep the result only if its a significant improvement. Avoid refactoring only because you are enamored of refactoring. (Or enamored of a rule.) Goes for any programming technique/tool, really.