News for hackers. Stuff that matters.
Re:Low Hanging Fruit of Programming Language Design by titzer on Wednesday November 22, 2017 @10:31AM (#15757498)
You know, sometimes duplicating code is just better. I've been around long enough to see many attempts at refactoring to share code! crash and burn. The hoops and complexity that some people go through to DRY (don't repeat yourself) are worse sometimes than the cost of maintaining two copies.
Re:Low Hanging Fruit of Programming Language Design by ajmurmann on Wednesday November 22, 2017 @11:02AM (#15757789)
Sandi Metz said in one of her talks that wrong abstractions are worse than duplication and we just tell new developers about duplication because it's the only thing they understand. Strongly agreed.
Re:Low Hanging Fruit of Programming Language Design by catnaroek on Wednesday November 22, 2017 @07:37PM (#15761929)
Most programmers wouldn't know an abstraction if it hit them in the face. In fact, I highly doubt that most existing technology stacks allow you to build any abstractions at all, in the original sense of functionality so self-contained that a user can't distinguish it from a programming language primitive, not just syntactically, but also semantically. In particular:

(0) If you need to know or care about its implementation details when you use it, it's not an abstraction.

(1) If unrelated ad hoc cases are hardcoded together into a single procedure, it's not an abstraction.

(2) If it needs fifteen tunable parameters, some combinations of which don't even make sense, it's not an abstraction.

At some point in time, programmers discovered that selection (if, switch, pattern mathcing, whatever) could be replaced with dynamically dispatched first-class procedure calls. Since then, they haven't stopped making their code more “abstract” by making their control flow impossible to follow. Alas, this isn't real abstraction, because it doesn't achieve the intended purpose of reducing the number of things you have to keep in your head simultaneously.

So, rather than blaming “wrong abstractions”, I'd blame “not real abstractions”.

Re:Low Hanging Fruit of Programming Language Design by com2kid on Wednesday November 22, 2017 @07:55PM (#15762001)
> In fact, I highly doubt that most existing technology stacks allow you to build any abstractions at all, in the original sense of functionality so self-contained that a user can't distinguish it from a programming language primitive

Two of the most popular languages out there, C++ and JavaScript, are both great at this.

Sometimes, too great.

Re:Low Hanging Fruit of Programming Language Design by catnaroek on Wednesday November 22, 2017 @08:00PM (#15762025)
Are you serious? How do I define complex numbers (just to give one example) in JavaScript that actually behave like, you know, complex numbers? Recall that:

(0) The usual symbol for complex number addition is +.

(1) The operation isn't associated to any individual complex number (i.e., it's not an object method), but is rather an intrinsic part of the algebraic structure of the complex numbers.

(2) Complex numbers don't have a physical object identity in memory.

C++ fares a little better than JavaScript, but not a lot better.

Re:Low Hanging Fruit of Programming Language Design by com2kid on Thursday November 23, 2017 @02:32AM (#15763482)
The major JS frameworks look like languages onto themselves. JavaScript lacks operator overloading, so there are some limits, but it is quite possible to turn JS into something very unrecognizable.

C++ templates allow for almost anything. I have seen incredibly strange things done to the language with templates. Since Templates can be used to parse, and execute, entirely different programming languages, it is hard to imagine a level of abuse to which C++ templates are not amenable.

Re:Low Hanging Fruit of Programming Language Design by catnaroek on Thursday November 23, 2017 @03:00AM (#15763587)
> The major JS frameworks look like languages onto themselves (...) it is quite possible to turn JS into something very unrecognizable.

That doesn't mean anything. With enough determination, anything can be turned into “something very unrecognizable”. Heck, sometimes the results are relatively pleasant even: http://libcello.org/ . But, if a language doesn't allow you to define basic abstractions like complex numbers (not to mention slightly more elaborate ones, like queues that are indistinguishable from one another if they contain the same elements in the same order), then that obviously reflects very poorly on its abstraction facilities.

> C++ templates allow for almost anything. I have seen incredibly strange things done to the language with templates.

What you can't do in C++, however, is make your abstractions not leak. Every template library is one template specialization away from being broken. This is unlike languages that enforce their abstractions by more robust means, like parametricity and macro hygiene.

Re:Low Hanging Fruit of Programming Language Design by stcredzero on Wednesday November 22, 2017 @01:09PM (#15758982)
wrong abstractions are worse than duplication

This needs to be given a catchy phrasing and spread all across developer communities!

WTD == Worse Than Duplication?

WAAW == Wrong Abstractions Are Worse?

Re:Low Hanging Fruit of Programming Language Design by jonahx on Wednesday November 22, 2017 @01:54PM (#15759391)
It's a great principle and I reference it frequently, but as a catchy buzzphrase it has the potential to do as much harm as good: it's easy to abuse as an excuse for copy paste programming.

Duplication is still bad, and correct abstractions are still good.

Re:Low Hanging Fruit of Programming Language Design by djur on Wednesday November 22, 2017 @02:27PM (#15759682)
See also "premature optimization is the root of all evil", which I occasionally see stretched to include any kind of thinking about the performance characteristics of your code, even down to basic design decisions like data structures or algorithms.
Re:Low Hanging Fruit of Programming Language Design by grandalf on Wednesday November 22, 2017 @05:19PM (#15761187)
Great point. I've found that most codebases have some abstractions that work very well and others that work less well, and simply having awareness of this can help make the code far more readable.

For instance, one might say: "The way x is implemented works, but it's not quite the right abstraction, here's why". That is way more informative than saying it's simply the wrong abstraction or considering it good enough and ignoring its deficiencies.

Re:Low Hanging Fruit of Programming Language Design by gcatlin on Wednesday November 22, 2017 @03:05PM (#15760091)
Premature abstraction is the root of all evil.
Re:Low Hanging Fruit of Programming Language Design by kirkules on Thursday November 23, 2017 @04:04AM (#15763854)
Premature abstraction is pure mathematics
Re:Low Hanging Fruit of Programming Language Design by jonahx on Wednesday November 22, 2017 @01:51PM (#15759364)
Re:Low Hanging Fruit of Programming Language Design by gowld on Wednesday November 22, 2017 @05:59PM (#15761438)
DRY is not the same as adding layers of abstraction.
Re:Low Hanging Fruit of Programming Language Design by leakyabs on Wednesday November 22, 2017 @01:52PM (#15759371)
I feel this in my soul. I work on a JavaScript project now where sometimes two duplicate pieces of code are “refactored” into one function unnecessarily. Just because there are two similar lines doesn’t mean they need to be their own function/class/whatever, especially when it makes it harder to generalize later when you need similar code that DOESN’T make sense to lump in with that abstraction.

I keep having to simplify the code and it’s exhausting.

Re:Low Hanging Fruit of Programming Language Design by kibwen on Wednesday November 22, 2017 @02:35PM (#15759779)
Agreed that fanatical adherence to DRY is a mistake. I did this when I was first starting out: any code that looked similar at all would be factored out into a new hyper-specific function, regardless of how unrelated the original pieces of code were. But this introduces extreme coupling between components that have no right being coupled, and makes evolving the code a nightmare (suddenly all your DRY-borne functions grow immense lists of configuration parameters to account for unforeseeable differences).

At the same time, let's not take this dogma too far in the other direction. Repeating oneself is still undesirable, but DRY-related refactoring should focus on behavior and intent, not on how the code looks. And abstractions are still good (they're how we get anything done at all!), but leaky abstractions should be avoided, indirection-via-abstraction should be minimized, and we should be vigilant against extreme and overeager overapplication of abstraction (insert your blub joke here).

Re:Low Hanging Fruit of Programming Language Design by gowld on Wednesday November 22, 2017 @05:59PM (#15761434)
> But this introduces extreme coupling between components that have no right being coupled, and makes evolving the code a nightmare (suddenly all your DRY-borne functions grow immense lists of configuration parameters to account for unforeseeable differences).

* A and B both depend on the same logic X.

* You make a shared helper function X

* Requirements for A change, necessitating a change to x.

* You check if B requires the same change. If yes, change X. If no, fork X into X_A and A_B

Now you're staying DRY, and DRY has saved the day by forcing you to think about whether A's change should impact B. If you had prematurely forked X, then X_A and X_B would drift without any sanity check on whether they should.

Re:Low Hanging Fruit of Programming Language Design by antonvs on Wednesday November 22, 2017 @04:14PM (#15760715)
Balancing these kinds of constraints - how much and when to DRY vs. copy - is a big part of what engineering is about.

It also needs to take into account factors like available budget of time and money, capabilities of the team working on the code, etc.

Re:Low Hanging Fruit of Programming Language Design by naasking on Wednesday November 22, 2017 @10:40AM (#15757600)
> The hoops and complexity that some people go through to DRY (don't repeat yourself) are worse sometimes than the cost of maintaining two copies.

The number of bugs are proportional to the lines of code, this is undeniable from empirical data. Ergo, fewer lines of code will tend to yield fewer bugs. So if your code is literally the same, there's no reason not to extract it into a function.

That said, the moment you have to start adding parameters in order to successfully factor out common code, ie. to select the correct code path taken depending on caller context, that's when you should seriously question whether the code should actually be shared between these two callers. More than likely in this case, only the common code path between two callers should be shared.

Re:Low Hanging Fruit of Programming Language Design by dllthomas on Wednesday November 22, 2017 @10:51AM (#15757686)
> So if your code is literally the same, there's no reason not to extract it into a function.

If the two pieces of code are likely to change in different ways, for different reasons, that is a strong reason not to extract it into a function even if they happen to be character-for-character identical for the moment.

Less code means fewer bugs, but that doesn't mean I should be working on the gzipped representation.

Re:Low Hanging Fruit of Programming Language Design by naasking on Wednesday November 22, 2017 @11:21AM (#15757983)
> If the two pieces of code are likely to change in different ways, for different reasons, that is a strong reason not to extract it into a function even if they happen to be character-for-character identical for the moment.

The future is much more malleable than your immediate needs.

But even if your future turns out to be true, the code you need to refactor is then already extracted to a function, so you can easily duplicate that function, make your localized changes, and change the callers to the new function. So this is still the best route.

> Less code means fewer bugs, but that doesn't mean I should be working on the gzipped representation.

Don't be absurd. gzipping doesn't preserve your program in human-readable form. Extracting code into a reusable function makes your program more human readable, not less.

Re:Low Hanging Fruit of Programming Language Design by dllthomas on Wednesday November 22, 2017 @12:45PM (#15758746)
In the parent comment, you said there was no reason. I maintain that it's a strong reason. It may or may not be a sufficient reason, weighed against other considerations. In particular, if it harms readability that's also a strong consideration.

But sometimes it can help readability, too. "DRY" as a principle was originally formulated in terms of repetition of pieces of knowledge rather than code, and I think in those terms it's far more useful. If this code represents "how we frob the widget" and that code represents "how we tweak the sprocket" and there's no reason for those to agree, they should probably be separate functions. Pulling them out into a "tweaking_sprockets_or_frobbing_widgets" function is making things less readable, because it's conflating things that shouldn't be conflated. If there is not some underlying piece of knowledge - some statement about the domain or some coherent abstraction that simplifies reasoning or some necessary feature of the implementation - combining superficially similar things is just "Huffman coding".

Re:Low Hanging Fruit of Programming Language Design by falcolas on Wednesday November 22, 2017 @11:39AM (#15758153)
> Extracting code into a reusable function makes your program more human readable

When done properly, yes. When done to the point where a five line function is created with ten inputs (yes, this is real), no. But DRY tells us that the five lines of duplication is unconditionally worse.

Hell, I've even seen things like logging/write tuples (i.e. log the error, write the to a socket) encapsulated, even though the only non-parameter code ends up being the two function calls.

Anything, taken to extremes is bad. The problem with DRY is it encourages that extremism.

Re:Low Hanging Fruit of Programming Language Design by dllthomas on Wednesday November 22, 2017 @01:06PM (#15758947)
> But DRY tells us that the five lines of duplication is unconditionally worse.

I agree that's often how DRY is understood, and that it can be a problem.

It is not how DRY was originally formulated, which was "Every piece of knowledge must have a single, unambiguous, authoritative representation within a system." This differs from blind squashing of syntactic repetition in two important ways. First, as under discussion here, if things happen to be the same but mean different things, combining them is not "DRY-er". Second, there can be repetition of knowledge without repetition of code. For instance, if we are telling our HTML "there is a button here", and our JS "there is a button here", and our CSS "there is a button here", we're repeating the same piece of knowledge three times even though the syntax looks nothing alike.

I make no claim as to whether the flawed, more common understanding or the original intent is what "DRY really means", but I think the latter is more useful.

Re:Low Hanging Fruit of Programming Language Design by zbentley on Wednesday November 22, 2017 @03:40PM (#15760406)
This is correct, and the below should not be taken in argument with it.

DRY as a guiding principle sometimes has a secondary beneficial effect that was not discussed. Two pieces of code that happen to be the same but "mean different things" should not automatically be deduplicated by dumb extraction. However, the fact that those two things share code may, when viewed through the lens of "prioritize-DRY-ness", hint that the two share a common underlying goal, which can be abstracted out into functionality that can be used by both.

Put another way: if the code to control a nuclear reactor circuit and the code to turn on a landing light on a plane happen to be the exact same, they shouldn't be blindly deduplicated into some library function, but the fact that they're the same may indicate a need for a more accessible, easily-usable-without-mistakes way of turning that kind of circuits on and off.

Re:Low Hanging Fruit of Programming Language Design by naasking on Wednesday November 22, 2017 @12:01PM (#15758337)
> When done properly, yes. When done to the point where a five line function is created with ten inputs (yes, this is real), no. But DRY tells us that the five lines of duplication is unconditionally worse.

I'm not convinced by your example. There are plenty of mathematical calculations taking numerous parameters that I think should be in a distinct function.

Even for non-mathematical calculations, 5 lines of code that are used repeatedly as some sort of standard pattern in your program should also get factored out. Like your logging example, ie. you consistently log everything in your program the same way, then sure, refactor that into a shared function. Then if you suddenly find you need to log more or less information, you can update it in one place.

Of course, I understand your meaning that sometimes factoring out doesn't make sense, but if you find repetition more than twice as per DRY, refactoring seems appropriate.

Re:Low Hanging Fruit of Programming Language Design by olau on Wednesday November 22, 2017 @12:14PM (#15758450)
I have another example.

In web frameworks, there is usually a little bit of boiler plate for each view.

You could refactor this completely away, but not without an almost total loss of flexibility and a good amount of readability too. Often the views will look very similar, then start diverging as the project grows.

With you on refactoring common patterns out, and yes, some people don't do this enough. But really, the important thing there is that those patterns are truly common to a large degree and should stay in sync - so it's worth it to introduce a maintain a new concept to keep them that way.

Re:Low Hanging Fruit of Programming Language Design by always_good on Wednesday November 22, 2017 @12:21PM (#15758529)
You DRY up code by writing indirection. That's the expense of all abstractions. You can't believe that all indirection is worth it at all costs, so I'm not sure what point you're belaboring.
Re:Low Hanging Fruit of Programming Language Design by naasking on Wednesday November 22, 2017 @02:16PM (#15759581)
> You can't believe that all indirection is worth it at all costs

I think I've been pretty clear about the costs and when this is worth it, particularly in my first post in this subthread, which I'll quote here:

> That said, the moment you have to start adding parameters in order to successfully factor out common code, ie. to select the correct code path taken depending on caller context, that's when you should seriously question whether the code should actually be shared between these two callers. More than likely in this case, only the common code path between two callers should be shared.

Or if you want a more concise soundbite: refactor if your indirection is actually a clear and coherent abstraction.

Re:Low Hanging Fruit of Programming Language Design by leakyabs on Wednesday November 22, 2017 @02:40PM (#15759836)
> When done properly, yes. When done to the point where a five line function is created with ten inputs (yes, this is real), no. But DRY tells us that the five lines of duplication is unconditionally worse.

I work with people I would describe as... junior at best (lots of boot campers) and I see this all the time. Functions that just return an anonymous function for no reason, half JSON blobs returned from functions that are called with one string instead of just repeating the blob in the code, etc.

Re:Low Hanging Fruit of Programming Language Design by coldtea on Wednesday November 22, 2017 @02:45PM (#15759885)
>The future is much more malleable than your immediate needs.

http://wiki.c2.com/?YouArentGonnaNeedIt

Re:Low Hanging Fruit of Programming Language Design by Chris_Newton on Wednesday November 22, 2017 @04:48PM (#15760997)
You Aren't Gonna Need It.

That’s a popular claim. I wonder how many failed projects could have it as their epitaph.

Have you ever worked on a project where the requirements changed so fundamentally from one day to the next that you truly, honestly had no idea where you were going next?

I haven’t. I’m not aware that I’ve ever met anyone else who has, either.

The claim that requirements always, or even usually, change so dramatically within such short timescales that it isn’t worth laying any groundwork a little way ahead simply doesn’t stand up to scrutiny, in my experience. Any project that was so unclear about its direction from one day to the next would have far bigger problems than how the code was designed.

Otherwise, there is always a risk that by being too literal, by ignoring all of your expectations about future development regardless of your confidence in them, you climb the mountain by climbing to one small peak, then down again and up the next slightly higher peak, and so on. This could be incredibly wasteful.

Of course requirements often change on real world projects. Of course I’m not advocating coding against some vaguely defined and mostly hypothetical future requirement five years in advance. But often you will have some sense of which requirements are going to be stable enough over the next day or week or month to base assumptions on them, and insisting on ignoring that information for dogmatic reasons just seems like a drain on your whole development process.

Re:Low Hanging Fruit of Programming Language Design by coldtea on Wednesday November 22, 2017 @04:58PM (#15761067)
>That’s a popular claim. I wonder how many failed projects could have it as their epitaph.

Way less than over-ambitious projects that died because of things that they didn't need, immortalized in lots of classic Comp-Sci literature. From Fred Brooks' books to Dreaming in Code:.

There's a reason it's a popular claim. In fact, popular means it's just repeated by many -- but this claim one can read repeated by the most experienced and revered programmers (or an analogous one, e.g. the KISS principle, "Do the simplest thing that could possible work", etc.), from the Bell Labs guys to the most celebrated programmers today.

>Have you ever worked on a project where the requirements changed so fundamentally from one day to the next that you truly, honestly had no idea where you were going next? I haven’t. I’m not aware that I’ve ever met anyone else who has, either.

Welcome to my life :-)

Not being snarky -- rapidly changing requirements is the number one complain in my kind of work.

Re:Low Hanging Fruit of Programming Language Design by Chris_Newton on Wednesday November 22, 2017 @05:19PM (#15761188)
Well, I didn’t say there was only one way a software project could fail! My point is simply that I believe anticipating and allowing for future requirements is a matter of costs and benefits. It’s about comparing the cost of making a wrong step and then having to backtrack with the cost of following a circuitous route to the final destination instead of a more direct one. Both are bad if we make the wrong choice, and we can’t see the future to make an informed decision about the right choice, but we can at least look at the expected cost either way and make an intelligent decision in any given case.
Re:Low Hanging Fruit of Programming Language Design by cgag on Wednesday November 22, 2017 @04:08PM (#15760654)
If it's not actually re-used then it's just making me jump around to see what's actually happening rather than reading straight through the code.
Re:Low Hanging Fruit of Programming Language Design by naasking on Thursday November 23, 2017 @02:25PM (#15767264)
> If it's not actually re-used then it's just making me jump around to see what's actually happening rather than reading straight through the code.

Firstly, you only actually create a function either when it is being reused, or because it's functionality is a logically separable responsibility and so you factor it out for understandability.

Either way, the function should also have a meaningful name describing its purpose so you don't have to jump around to understand what's actually happening.

Re:Low Hanging Fruit of Programming Language Design by sheepmullet on Wednesday November 22, 2017 @04:42PM (#15760943)
> The future is much more malleable than your immediate needs.

In practice I'm not really sure what you mean?

A good rule of thumb on large scale software projects is complexity begets complexity.

It then takes a lot of effort to return to simple code.

> Extracting code into a reusable function makes your program more human readable, not less.

Is that a given?

I've come across plenty of small functions that I couldn't understand without checking the calling functions for context.

Re:Low Hanging Fruit of Programming Language Design by naasking on Thursday November 23, 2017 @02:32PM (#15767297)
> In practice I'm not really sure what you mean?

Meaning, your future needs are ever changing and often unclear. Your present needs are immediate and usually obvious. Meet your present needs first and foremost without sacrificing flexibility to meet future needs. Factoring code into functions accomplishes this.

> I've come across plenty of small functions that I couldn't understand without checking the calling functions for context.

Sure, happens to me too when I don't assign meaningful names, or the functions don't actually encompass a small, meaningful set of responsibilities, or the functions use deep side-effects that require reasoning within larger contexts.

The problem with such programs isn't factoring into functions though. If anything, this step reveals latent structural problems.

Re:Low Hanging Fruit of Programming Language Design by gowld on Wednesday November 22, 2017 @06:03PM (#15761469)
> Less code means fewer bugs, but that doesn't mean I should be working on the gzipped representation.

That's a false analogy. gzip is non-syntax-preserving

Re:Low Hanging Fruit of Programming Language Design by fauigerzigerk on Wednesday November 22, 2017 @11:41AM (#15758175)
>The number of bugs are proportional to the lines of code, this is undeniable from empirical data.

Do you have a link to this undeniable data? I haven't seen many empirical code quality studies that are not littered with possible confounders. It's very difficult to do these studies.

I do believe that bug density is roughly proportional to the size of the codebase if you average over large corpera of code. But the bug density of different types of code within those corpera varies a great deal in my experience.

So I think the important question is what sort of code is duplicated and why it is duplicated. Removing duplication means creating dependencies. If we create the right dependencies, i.e. the ones that enforce important invariants, that's a good thing. But that is a big if.

Re:Low Hanging Fruit of Programming Language Design by jack9 on Wednesday November 22, 2017 @12:37PM (#15758672)
You will enjoy the video in its entirety, if you haven't seen it.

https://vimeo.com/9270320 ~37:00 he covers:

El Emam et al (2001): The Confounding Effect of Class Size on the Validity of Object-Oriented Metrics

http://ieeexplore.ieee.org/document/935855/

Re:Low Hanging Fruit of Programming Language Design by naasking on Wednesday November 22, 2017 @11:54AM (#15758280)
> Do you have a link to this undeniable data?

For a summary, sure [1]. There have been loads of studies on various metrics, but none have actually been any better than simple lines of code, despite the fact that it has such a large variance as a metric.

> Removing duplication means creating dependencies. If we create the right dependencies, i.e. the ones that enforce important invariants, that's a good thing. But that is a big if.

While enforcing invariants would certainly be good, I'm not convinced that's the only reason DRY reduces bugs. Common functions get manual reviews every time the code that calls them also get reviewed and/or refactored, whether due to new features or bugfixes.

DRY increases exposure of more commonly used paths through your program.

[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5544018/

Re:Low Hanging Fruit of Programming Language Design by fauigerzigerk on Wednesday November 22, 2017 @12:38PM (#15758682)
>There have been loads of studies on various metrics, but none have actually been any better than simple lines of code

True, but that doesn't mean SLOC is a very useful metric. Say you were to rewrite a large Java codebase in a language that eliminates all getters and setters. You will have greatly reduced the number of SLOC, but it is very unlikely that you will have reduced the number of bugs very much.

In other words, going for the low hanging fruit of programming language design wouldn't necessarily help much. Bugs are not evenly spread out over the entire codebase.

Re:Low Hanging Fruit of Programming Language Design by nkurz on Wednesday November 22, 2017 @01:01PM (#15758887)
You will have greatly reduced the number of SLOC, but it is very unlikely that you will have reduced the number of bugs very much.

I agree strongly with your point about potential confounders (and was about to make it myself) but now you are making your own assertion that I'm uncertain about. Why are you confident that switching to a language that greatly reduces the number of lines of code would not reduce the number of bugs?

While I don't know of any hard evidence, it passes my internal plausibility test that if some of the "2 screen" functions become "1 screen" function, bugs might be less likely. There might be some counter-force that would confound this, but I wouldn't eliminate it out of hand. So what makes you say "very unlikely" rather than "not necessarily true"?

Re:Low Hanging Fruit of Programming Language Design by dllthomas on Wednesday November 22, 2017 @01:18PM (#15759067)
I think the idea was "A language that is otherwise identical to Java but eliminates the need to write trivial getters and setters." This obviously reduces line count, but probably does not equivalently reduce bug count, as the removed lines are very unlikely to contain bugs.
Re:Low Hanging Fruit of Programming Language Design by nkurz on Wednesday November 22, 2017 @02:17PM (#15759585)
This obviously reduces line count, but probably does not equivalently reduce bug count, as the removed lines are very unlikely to contain bugs.

While I agree that the bugs are not likely to be in the trivial code, I don't think it's a given that presence of the trivial code has no impact on the number of bugs elsewhere. Consider a "fatigue" based model, where the human brain is distracted by the monotony of the bug-free getters and setters and thus unable to pay sufficient attention to the logic bugs elsewhere in the program. And again, I'm not making that claim that eliminating boilerplate reduces bugs, only objecting to the assumption that it does not.

Re:Low Hanging Fruit of Programming Language Design by dllthomas on Wednesday November 22, 2017 @02:32PM (#15759736)
My intent was to clarify (what I perceived to be) the parent's argument, more than make one of my own.

I think if our process is "1) write the software in Java, 2) remove those lines", it's clear that we've probably changed the average bug density of the project. I agree that there is much reason for concern in generalizing that result to what would have happened if we'd written in that other language to begin with.

Re:Low Hanging Fruit of Programming Language Design by fauigerzigerk on Wednesday November 22, 2017 @02:04PM (#15759475)
I simply haven't found many bugs in (mostly auto-generated) getters/setters during the past 25 years, but it's purely anecdata of course.

>While I don't know of any hard evidence, it passes my internal plausibility test that if some of the "2 screen" functions become "1 screen" function, bugs might be less likely.

Yes, mine too, but only for randomly chosen pieces of code. What I don't believe is that the linear correlation between SLOC and bugs that studies have found in large codebases allows us to pick and choose the lines of code that are easy to eliminate and expect the number of bugs to drop proportionately.

Re:Low Hanging Fruit of Programming Language Design by geofft on Wednesday November 22, 2017 @12:06PM (#15758381)
> The number of bugs are proportional to the lines of code, this is undeniable from empirical data. Ergo, fewer lines of code will tend to yield fewer bugs.

I do not think that this analysis can be applied to decisions about whether to duplicate code or not.

First, if your code is the same, you're not going to have two different bugs in the two copies of it.

Second, trying to change the number of lines of code in your project without making deeper changes is the sort of thing that very directly confounds the analysis. It's like going from "Smaller companies have happier employees" to "So we should fire half our employees because empirical data shows the rest will instantly become happier."

Re:Low Hanging Fruit of Programming Language Design by naasking on Thursday November 23, 2017 @02:34PM (#15767311)
> First, if your code is the same, you're not going to have two different bugs in the two copies of it.

No, but you might see two different buggy behaviours due to contextual differences in how the code is used.

> Second, trying to change the number of lines of code in your project without making deeper changes is the sort of thing that very directly confounds the analysis.

Except refactoring into smaller reusable functions is precisely a deep structural change.

Re:Low Hanging Fruit of Programming Language Design by geofft on Thursday November 23, 2017 @05:24PM (#15768208)
The comment I was replying to was about deduplicating code that is "literally the same". In that case, you wouldn't make the code more or less buggy if you collapsed the multiple copies of the code down into one, and no deep structural changes would be involved in refactoring.

If there are actual changes involved in refactoring, I have no a priori expectation of whether that reduces or increases bug count, and I can see good arguments that it's likely to increase them, since you're making the code more complex in order to satisfy the demands of multiple consumers and therefore exposing each consumer's unique complexity to the other as bug surface. (Case in point: Heartbleed resulted entirely from a little-used extension to DTLS, a variant of TLS over UDP, which 99+% of OpenSSL users never cared about.)

Re:Low Hanging Fruit of Programming Language Design by naasking on Friday November 24, 2017 @11:59AM (#15771771)
> The comment I was replying to was about deduplicating code that is "literally the same". In that case, you wouldn't make the code more or less buggy if you collapsed the multiple copies of the code down into one, and no deep structural changes would be involved in refactoring.

Firstly, any refactoring involves a code review of what you're factoring out. This has a non-zero probability of revealing bugs, so I already disagree with your claim that it wouldn't change the bug count.

Secondly, if you're having difficulty refactoring, that's a strong hint at deeper structural problems, so it yields information on what kinds of structural changes are needed.

> If there are actual changes involved in refactoring, I have no a priori expectation of whether that reduces or increases bug count, and I can see good arguments that it's likely to increase them, since you're making the code more complex in order to satisfy the demands of multiple consumers.

Are you making it more complex? Because that doesn't seem like a sound refactoring in my mind. Special cases require special code, you don't place special cases in a general function, unless the function itself only handles the special case. I already covered this in my original post where I discuss when DRY isn't appropriate.

> (Case in point: Heartbleed resulted entirely from a little-used extension to DTLS, a variant of TLS over UDP, which 99+% of OpenSSL users never cared about.)

I don't see how this is a point in your favour. It's a point that little used and little inspected paths are more likely to be vulnerable. But a reused function gets more use and more review than inline code. In other words, Heartbleed would probably still not have been found if it weren't part of a common function, and instead were littered in various places throughout a code base.

Re:Low Hanging Fruit of Programming Language Design by coldtea on Wednesday November 22, 2017 @02:50PM (#15759933)
>The number of bugs are proportional to the lines of code, this is undeniable from empirical data. Ergo, fewer lines of code will tend to yield fewer bugs.

The first part is obvious (more code of course brings the potential for more bugs).

The deduction (hence more code = more bugs) is only valid if the studies controlled for the similarity of the code, cyclomatic complexity and other such factors.

Just because more code has impact in the number bugs, doesn't mean more code by itself means more bugs. Correlation != causation.

Program A having more code than program B could mean e.g.:

(1) A is inherently more complex (and really needs more lines, the same way an IDE needs more lines than an simple text editor), and thus will naturally be more prone to bugs.

(2) A and B has similar functionality as programs, but B is written by people who meander and write bloated code and needless abstractions (turning a 100 line program into a 1000 line hell of "design patterns" and factorySingletonProxy "flexibility"). Which will again bring in more bugs.

But that's not necessarily the case if A is bigger than B due to simple repetition that doesn't introduce complexity. Which is exactly what we're discussing in this thread.

For a trivial example, 10.000 lines of "print 'hello world'" repeated won't have more bugs than a 1000 line complex C program.

Re:Low Hanging Fruit of Programming Language Design by naasking on Wednesday November 22, 2017 @04:24PM (#15760798)
> Just because more code has impact in the number bugs, doesn't mean more code by itself means more bugs. Correlation != causation.

So the only possible causations for the correlation we're discussing are:

1. more program code causes more bugs

2. more bugs causes more program code

3. some unknown third factor(s) simultaneously causes both more bugs and more program code

I think 1 and 3 are most often the case, where 3 could be something like developer inexperience, although some studies have shown that even experienced developers still introduce bugs at comparable rates to novices (just lower constant factors). I think 2 sometimes happens to address immediate needs, ie. hotfix for specific bug X may introduce more bugs, but I doubt it's the rule.

Regardless, my original claim still seems pretty undeniable, ie. more program code tends to yield more bugs.

> For a trivial example, 10.000 lines of "print 'hello world'" repeated won't have more bugs than a 1000 line complex C program.

But 10,000 lines of "print 'hello wrld'" would have more bugs than a 1,000 line complex C program. Probably on the order of 9,000 more bugs in fact.

The numbers we're talking about here are averages across all programs of comparable length, not to be applied literally to any specific program, because it turns out that those specific program qualities don't really matter, ie. LOC is still a more accurate predictor of bug count than cyclomatic complexity and other metrics.

Thus I can say that a 1,000 line program probably has about X bugs, and I probably won't be off by an order of magnitude unless the program was verified by a theorem prover or something along those lines. Something like verification is really the only confounder that I've come across.

Re:Low Hanging Fruit of Programming Language Design by bmurphy1976 on Wednesday November 22, 2017 @11:24AM (#15758025)
> The number of bugs are proportional to the lines of code, this is undeniable from empirical data. Ergo, fewer lines of code will tend to yield fewer bugs. So if your code is literally the same, there's no reason not to extract it into a function.

Code sharing often increase coupling and the coupling can be at odds with naive attempts at achieving DRY. The real world is not as simple as this makes it out to be.

You are right more code means more bugs, but more code might also be the difference between a viable product and one that hasn't been built yet because everybody is locked into DRY bureaucratic hell. As in everything, there's a balancing act that needs to be performed and that requires judgement and experience.

Re:Low Hanging Fruit of Programming Language Design by ModernMech on Wednesday November 22, 2017 @11:20AM (#15757980)
> So if your code is literally the same, there's no reason not to extract it into a function.

To a point. Abstraction also creates indirection, which can make debugging harder, especially for people unfamiliar with your code.

Re:Low Hanging Fruit of Programming Language Design by nitely on Wednesday November 22, 2017 @11:44AM (#15758204)
> The number of bugs are proportional to the lines of code, this is undeniable from empirical data.

Including unit tests? A code base with unit test is likely to contain less bugs and more lines of code. And what about code golf?. This just does not seem right and I believe the original saying is related to languages with/without rich stdlibs and/or the NIH syndrome.

Re:Low Hanging Fruit of Programming Language Design by sethev on Wednesday November 22, 2017 @01:51PM (#15759359)
>The number of bugs are proportional to the lines of code, this is undeniable from empirical data. Ergo, fewer lines of code will tend to yield fewer bugs. So if your code is literally the same, there's no reason not to extract it into a function.

Except you've added a dependency that connects two pieces of code that were independent before.

Re:Low Hanging Fruit of Programming Language Design by rapala on Wednesday November 22, 2017 @02:31PM (#15759727)
It depends. If the duplicated code is supposed to do the same thing in both places then the two pieces of code are already dependent. You change one of the duplicates, you need to change the other one two.
Re:Low Hanging Fruit of Programming Language Design by vidarh on Wednesday November 22, 2017 @01:52PM (#15759366)
The number of bugs are proportional to the lines of code, but shared code can massively multiply the cost of making a change, because of the risks involved.

Sometimes duplication is worth it simply because it reduces the impact of changes.

Re:Low Hanging Fruit of Programming Language Design by laythea on Wednesday November 22, 2017 @01:22PM (#15759101)
If code block A has X bugs and code block B=A. Then a code block containing B+A still contains X bugs.
Re:Low Hanging Fruit of Programming Language Design by AnimalMuppet on Wednesday November 22, 2017 @01:36PM (#15759217)
But if I fix all X bugs in code block A, then a code block containing B+A still contains X bugs.

That's one of the big problems with code duplication. "You fixed a bug? Great. Did you fix it everywhere?" It's better if there's only one place to fix it, because when you've fixed it, you've fixed all of it.

But as others have said, eliminating this is not the only good thing in programming. There are limits to how far you should go to prevent or eliminate duplication.

Re:Low Hanging Fruit of Programming Language Design by naasking on Wednesday November 22, 2017 @02:28PM (#15759692)
> It's better if there's only one place to fix it, because when you've fixed it, you've fixed all of it.

You've got one aspect, and the other aspect is that code factored into a reused function F is now reviewed more too, ie. whenever you're reviewing the callers, you work through the functions they call as well to trace the behaviour. So you're also much more likely to find any bugs in F.

It's a double-whammy for bug squashing, which is why DRY is such an important principle. Certainly there are some misapplications, but its benefits hold up pretty well in a wide variety of scenarios.

Re:Low Hanging Fruit of Programming Language Design by whathaschanged on Wednesday November 22, 2017 @07:13PM (#15761804)
If the bug being fixed was expected behavior by all the other dependencies, you just screwed over everyone else who should have duplicated what they needed in the first place.
Re:Low Hanging Fruit of Programming Language Design by amorphid on Wednesday November 22, 2017 @01:10PM (#15758988)
> So if your code is literally the same, there's no reason not to extract it into a function.

Isn't that how we ended up with left-pad.js?

Re:Low Hanging Fruit of Programming Language Design by naasking on Wednesday November 22, 2017 @02:24PM (#15759642)
Nothing wrong with importing a left-pad module if you're using a sane language with a sane development toolbox. I'll leave it to you to decide whether JS fits that bill.
Re:Low Hanging Fruit of Programming Language Design by fooblitzky on Wednesday November 22, 2017 @01:49PM (#15759342)
These days I prefer SPOT (single point of truth) over DRY. I find DRY tends to encourage people to use hideously complicated abstractions to avoid code that even looks similar. It's also way easier to refactor some duplicated code than to try and untangle complicated abstractions.

SPOT more accurately identifies the pain point of duplicated code - if there is an algorithm or configuration value that needs to be specified, than that should just be in a single place, everything consistent and easy to change.

Re:Low Hanging Fruit of Programming Language Design by gowld on Wednesday November 22, 2017 @06:02PM (#15761453)
https://en.wikipedia.org/wiki/Don%27t_repeat_yourself

> The DRY principle is stated as "Every piece of knowledge must have a single, unambiguous, authoritative representation within a system".

How is SPOT different?

Re:Low Hanging Fruit of Programming Language Design by dllthomas on Wednesday November 22, 2017 @08:10PM (#15762068)
I'm unclear as to whether the parent is preferring the name SPOT for the same concept, or preferring what they are presenting as a different concept (which, as you note, seems to match the original definition of DRY).
Re:Low Hanging Fruit of Programming Language Design by zaarn on Thursday November 23, 2017 @05:43AM (#15764228)
I think SPOT is different in that DRY has become misunderstood, so the "Single Point of Truth" brings the proper point across a bit better.
Re:Low Hanging Fruit of Programming Language Design by HelloNurse on Wednesday November 22, 2017 @05:03PM (#15761093)
Good name, it helps with distinguishing things that are meant to be the same (e.g a single implementation of an encryption algorithm, used in various places) and things that are distinct but accidentally identical (e.g. permutations in different stages of that algorithm).
Re:Low Hanging Fruit of Programming Language Design by andrewingram on Wednesday November 22, 2017 @03:27PM (#15760289)
The question I ask myself is: Are these two things that look the same, actually the exact same problem, and do I expect them to continue being the exact same problem?

If the answer is an easy yes, then build your abstraction. If it's no, don't. If it's maybe, leave the duplication in place until you have a clearer understanding.

Re:Low Hanging Fruit of Programming Language Design by novembermike on Wednesday November 22, 2017 @01:18PM (#15759069)
One of the problems is that people hear "DRY" and they don't hear "Every piece of knowledge must have a single, unambiguous, authoritative representation within a system", which is the actual idea. They don't even share any words. The actual idea is really cool and demands correct abstractions, DRY is about making things not look the same.
Re:Low Hanging Fruit of Programming Language Design by kaeluka on Wednesday November 22, 2017 @10:38AM (#15757572)
Code duplication can be seen as a "defensive copy". If you duplicate code, the original version can be changed without breaking the copies.

Disclaimer: I'm not making a value statement here :-)

Re:Low Hanging Fruit of Programming Language Design by sanderjd on Wednesday November 22, 2017 @11:03AM (#15757805)
In an ideal world, what you want is more like copy-on-write. Share the code until one of the clients of that code needs a change that the others do not, then copy it. In reality, what happens more often is an accumulation of different functionality used by different clients. Copying "defensively" as you suggest can help avoid that problem, but it has the opposite problem of requiring changes to all copies when all clients want the same thing. In my experience, this is the problem more likely to lead to painful mistakes. There is no silver bullet...
Re:Low Hanging Fruit of Programming Language Design by usrusr on Wednesday November 22, 2017 @04:20PM (#15760763)
Even with "copy on write" there would still be the problem of wether to copy before write or not: requirements for usage A change because someone recently thought hard about that part. Usage B is most likely not on anybody's mind right now andchanging that is quite some effort. It's a problem no matter wether you copy, DRY or something in between.

Maybe some of our processes, language and experience from the version control space (branch, merge, rebase, upstream...) could be ported over to the reuse vs duplication problem. But when you layer that with the conventional use of those concepts in version control, the result would be quite the multidimensional mindbender. With that as a hypothetical baseline, simply guessing a good compromise between abstraction and duplication and then dealing with the consequences suddenly does not seem that bad a fate.

Re:Low Hanging Fruit of Programming Language Design by goatlover on Wednesday November 22, 2017 @03:27PM (#15760296)
You mean like prototype-based inheritance?
Re:Low Hanging Fruit of Programming Language Design by zbentley on Thursday November 23, 2017 @07:38AM (#15764620)
I don't think prototype-based inheritance (or monkeypatching) is any better at avoiding duplication than "regular" Java-style inheritance, and both will only help avoid certain kinds of duplication some of the time.

If you need to modify a small part of a class's function, there's typically no way in a prototype/traditional inheritance language, to say "like the function from the parent class, but with these 5 lines different".

Inheritance of any kind is a handy tool to help with the "whole class files full of pages of identical code with one different method" problem, but a) that's not the cause of a lot of duplication out there, b) you still have to know/care to implement the inheritance, and c) it can come with its own struggles, a la tight coupling to implementations/classes you may not fully control. This isn't a pro-OO or anti-OO screed; just pointing out that inheritance as a means to DRY up code is a limited solution at best.

Re:Low Hanging Fruit of Programming Language Design by sanderjd on Wednesday November 22, 2017 @06:57PM (#15761747)
I don't think so... I'm talking about copying the code itself.
Re:Low Hanging Fruit of Programming Language Design by whathaschanged on Wednesday November 22, 2017 @07:23PM (#15761843)
So ctrl-f and ctrl-v using the diff as needed?

I wonder if the man hours wasted on hunting through someone else's code to figure out the bajillion abstracted functions are doing when you are trying to make a change outweigh the minor inconvenience of copying and pasting changes are? Yes the latter introduces more opportunities for errors, but are they more than knowing all your dependencies and ensuring your changes don't break their expectations? Unit tests are only as useful as the man who knows the future.

Re:Low Hanging Fruit of Programming Language Design by sanderjd on Sunday November 26, 2017 @03:07PM (#15782709)
That's basically what I'm saying: it's unknowable whether copying and pasting and needing to keep all the copies up to date in the future will result in more or less work and bug fixing than re-using code. I think the best solution is for experienced developers to make an educated guess on a case by case basis. But you can't write that into a generic principle.
Re:Low Hanging Fruit of Programming Language Design by kaeluka on Wednesday November 22, 2017 @01:18PM (#15759072)
> In an ideal world, what you want is more like copy-on-write. Share the code until one of the clients of that code needs a change that the others do not, then copy it.

Good point. The problem with that is, of course, that you may not be aware of others existing.

> There is no silver bullet...

+1

Re:Low Hanging Fruit of Programming Language Design by quotemstr on Wednesday November 22, 2017 @12:47PM (#15758760)
I like to say that programming is not an exercise in Huffman compression. Not all structurally similar chunks of code should be unified: sometimes these chunks are _semantically_ different, and unifying them would create a false and confusing commonality.
Re:Low Hanging Fruit of Programming Language Design by nikbackm on Wednesday November 22, 2017 @12:53PM (#15758821)
That's what copy-on-write is for. ;)
Re:Low Hanging Fruit of Programming Language Design by pault on Wednesday November 22, 2017 @11:12AM (#15757896)
The issue comes from trying to DRY the code before the requirements and functionality are well defined. If you're doing major refactoring before you have something in production it's just premature optimization.
Re:Low Hanging Fruit of Programming Language Design by gnuvince on Wednesday November 22, 2017 @01:01PM (#15758893)
I mentioned a few months ago in a similar thread that I now actively try to recognize when I'm creating an incomprehensible 300-line "abstraction" in order to avoid copying four line of code.
Re:Low Hanging Fruit of Programming Language Design by usrusr on Wednesday November 22, 2017 @04:30PM (#15760843)
Did you ever succeed? ;)

I sometimes notice but just can't stop it. The next step is usually a desperate attempt of painting that pig with documentation.

Re:Low Hanging Fruit of Programming Language Design by breatheoften on Wednesday November 22, 2017 @07:26PM (#15761862)
The flip side of this occurs when trying to evolve systems with too much copying — particularly when there is not a good way to find where “these things should behave the same” has spread ... I get really nervous when coding up a pattern based on copying when the main reason for the copying is that there _isnt_ a good way to share the logic without a lot of hoops ... copying when I see how I could easily have shared the logic feels like such a safer decision, much easier to justify because I can always see how to refactor later if it proves that the copied pattern really does represent a use that evolves with shared requirements. When I’m put into a situation where I know that I’m only copying because I can’t see a good way for the logic to be shared I feel like I’m setting up an infinite game of bug wack-a-mole as errors due to untraceable synchronization requirements are doomed to reoccur forever as the system changes ... I find these scenarios frequently popping up when using tools with poor composition capacity (lots of build and deployment systems) — “I don’t know how to compose this stuff so I’m just going to copy and will hate this later” is the worst feeling.
Re:Low Hanging Fruit of Programming Language Design by pqh on Wednesday November 22, 2017 @05:27PM (#15761233)
Don't abstract it out until it's used in multiple places. Don't abstract it out using a complicated system, like inheritance, when you can use a function in a module. Don't abstract it out if it's superficially similar as opposed to actually the same thing.

Regarding the last point: iterators, good abstraction; averaging function, good abstraction. These things never change. But bad abstractions copy things that are only similar in appearance.

Re:Low Hanging Fruit of Programming Language Design by ericmcer on Wednesday November 22, 2017 @04:16PM (#15760734)
Programmers seem allergic to the idea that they would have to go through and update several functions that do the same thing. This is mentally a very trivial task though, much preferable to trying to alter some 'clever' dependency. In a super agile environment especially I find DRY practices will quickly build a huge web of garbage.
Re:Low Hanging Fruit of Programming Language Design by laythea on Wednesday November 22, 2017 @01:20PM (#15759086)
I've seen this also. The amount of hoops and jumps people go through in order to abstract something can be ridiculous, ending in code only the author can understand. Bad for business!
Re:Low Hanging Fruit of Programming Language Design by usrusr on Wednesday November 22, 2017 @04:40PM (#15760915)
...that only the author can understand, and not after being done writing. But still, good abstractions are much better than duplicated code, yet bad abstractions can easily be worse.

"Minimize both duplication and bad abstractions" should be something everybody can get behind, even if only because it leaves all the details open.

Re:Low Hanging Fruit of Programming Language Design by dragonwriter on Wednesday November 22, 2017 @04:43PM (#15760948)
> "Minimize both duplication and bad abstractions" should be something everybody can get behind, even if only because it leaves all the details open.

Leaving all the details open means the slogan is content-free. Why would anyone get behind it?

Re:Low Hanging Fruit of Programming Language Design by usrusr on Wednesday November 22, 2017 @07:18PM (#15761824)
Usually it happens when people put their favorite ideas into the blanks and do not realize that others could fill them in a different way.
Re:Low Hanging Fruit of Programming Language Design by geofft on Wednesday November 22, 2017 @12:08PM (#15758402)
r0ml gave a good argument for duplication over abstraction in his Debconf talk this year: https://debconf17.debconf.org/talks/194/ (video, I'm not sure if there's a transcript, but if you're bored on a Thanksgiving, I would recommend watching the whole thing)
Re:Low Hanging Fruit of Programming Language Design by mpweiher on Wednesday November 22, 2017 @04:38PM (#15760896)
"Premature abstraction is the root of all evil" -- me

On the other hand, maybe it also points to problems with our abstraction mechanisms?

Re:Low Hanging Fruit of Programming Language Design by justicezyx on Wednesday November 22, 2017 @10:57AM (#15757742)
I am pretty sure the crgoculting testless bloated code dismayed the incompetent refactorer. This is not a proof of benefit of duplication, it's a proof of the low quality of the legacy code.