This 90% of code is not genuine word-for-word boilerplate (copy/pasted from a known good source). This code is typically constructed fresh each time; or worse, copied from somewhere similar and quickly tweaked for names/types! (I do it, and I see it done all the time.)
I expect that the remaining 10% non-boilerplate code, taking 80% of the effort, is much more carefully considered, and less likely to contain those clumsy/forgetful off-by-one or buffer-overflow bugs.
Most people don't like writing that boilerplate once they know how to do it and have done it a few times, and would rather just call a function do_that_thing_need_done_on(input1, input2).
If it can't be factored out like that and is actual language boilerplate beyond a few lines, that's a failure of the language.
If these AI models are suggesting the code that could be called in a library/module instead of the the code to actually include and call a well known and trusted library or module, I'm not sure that's progress. At least when someone notices a bug or better way to do it and updates that module or library, consumers of that module can update and benefit from it, or at a minimum see that there were bugs in the version they're running they might want to address at some point.
I think the judgement call of when to use a library & what library to use is quite subjective, even for humans to get right.
If I'm doing JSON deserialisation it might suggest I use Gson library which would be much better than rolling your own. But the original authors are saying that you should prefer Moshi over Gson — I think it'd be hard for an AI to reach that conclusion though (though maybe not if it's doing something like tracking migrations in OS projects from Gson->Moshi).
With something a little more trivial — I don't want it to add in a dependency on left-pad, even though it has 2.5M weekly downloads so is arguably both well-known and trusted :)
You could probably set a threshold for how complex code is before it's suggested to be swapped out for a lib, but then is my code simple because I'm ignoring edge cases I should support, or because I've trimmed the fat on what I'm choosing to support (e.g. i18n, date handling, email validation etc.)
The only thing worse than using a module that has a bug/security problem for a function that's just a few lines and not used again in the codebase is when the content of that function is copied in place instead of being included and nobody has an easy way of knowing whether that's the code that was suggested and included in their project. Worst of both worlds.
A lot of programming languages have a lot of room for improvement though.
The skill is in selecting the optimal amount and form of redundancy.
For example, the typical ; statement terminator is redundant. People often ask, since it is redundant, why not remove it? People have tried that, and found out that the redundancy makes for far better error detection.
I think a better example is CoffeeScript, where almost any string is a valid program but probably not the one you wanted.
They usually have another statement terminator instead, like a newline.
Of course, the downside is that now you have way more tokens you need to know to understand some code, similarly to how Haskell code tends to have tons of mega-abstract function combinators or whatever whereas Go code is very simple. Which one is more readable depends on the reader, because a language like Go requires the reader to sift through more details, whereas Haskell is much terser but also requires much more pre-existing knowledge to understand.
I'm wondering if we can use these code-generation models to find "low entropy code" that is a prime target for turning into libraries.
A()
B()
C()
So someone thinks, hey, lets extract that bit, and create a function that does all three things def ABC():
A()
B()
C()
Now the calls site is much simpler and there is no more repetition: ABC()
Except you realise that in some call sites the B() call was missing, so to fix that you add an argument: def ABC(doB):
A()
if(doB):
B()
C()
The call site now looks a bit less clean, but still no repetition! ABC(True)
or ABC(False)
Then you realise in some cases the C() call is not necessary, so just add another argument: def ABC(doB, doC):
A()
if (doB):
B()
if (doC):
C()
And your call sites look like this: ABC(True, False)
etc.At that point the repetition in the original is definitely preferrable...
def handleNewEmail(isInboxDisplayed, areNotificationsEnabled):
AddToInbox();
if (isInboxDisplayed):
RefreshInboxView();
if (areNotificationsEnabled):
SendSystemNotification();
Not defending needless refactoring in all cases, it's definitely a judgement call.(Edit: formatting)
I'm talking about the case where the conditionals are only needed because of the refactoring.
To be honest, I find it hard to discuss readibility without a real-world example. The algebraic placeholders feel too terse.
def func1():
...
A()
B()
C()
...
def func2():
...
A()
B()
C()
...
Which is then refactored to def func1():
...
ABC()
...
def func2():
...
ABC()
...
def ABC():
A()
B()
C()
Which is sensible, but then other functions show up: def func3()
...
A()
C()
...
And you refactor to: def func1():
...
ABC(True)
...
def func2():
...
ABC(True)
...
def func3()
...
ABC(False)
...
def ABC(doB):
A()
if (doB):
B()
C()
I'll leave the final example to your imagination, since I've already used so much screen space.I'm currently in the process of undoing a lot of such refactorings in my codebase because it has become unmanageable.
I really think repetitive code is much easier to work with than overabstracted code.
My personal approach to combat this is better data structures that model the problem (and are checked by the compiler). Once this is in place I try to "flatten" the calls, so that things are mostly at the top level, or few levels deep, which usually comes up naturally once the data structure is consciously defined.
I try to have as much code as possible be "structure in - structure out" (pure functions) and to concentrate stateful code to work on the structure's fields/values. This is surprisingly easy once the data structures match the problem, instead of only growing organically.
Both options can be valid. DRY is a tangential topic, the main goal should be to keep your code as close to your mental model as possible (also keeping mental models in sync between team members, which is quite hard).
ABC being a sequence could be a pattern, or it could be a coincidence. You can't know which one it is just by looking at the code. The knowledge whether it's a sequence or not is in the business domain and in your mental model of that domain.
You might think you're disagreeing on DRY with another team member, but in reality you two have different mental models, and one of you is using DRY to justify a his.
"You seem to be using ifs and fors very much. These should be abstracted into a function."
Since this is HN I'm sure someone will say that the answer is Lisp. But can we do better?
Lisp may be an improvement.
Another issue is consistency. Take C, Javascript, or Go. Many loops are of the form
for (var i = 0; i < n; i++) { ... }
You could argue that "for i < n" provides the same information, but then you'd have to find a way to start the loop at a different offset, use a different end condition, or different "increment".But the situation is not that bad, is it? A few characters too many, so be it. I find reusability a much larger problem.
Yes programming language design is a social science (imo)
Sounds like tokenizer -> Markov chain? Surely something trained on a TPU is more sophisticated than something we could have done in the middle of the 20th century?
At some point the variations mean that more abstractions don't really help.
A string of length N is vastly more likely to be a valid Perl program than a valid Python program. Ultimately this meant that Perl programs, while easier to type, were much harder to read, and extremely easy to misinterpret.
Perl is not only hard to read because there are many shortcuts that might look like line noise for the inexperienced (hell, Rust have a bunch of those too), it's because there is a bunch of the ways to do anything.
Like take humble array
Perl> @a
$VAR1 = 1;
$VAR2 = 2;
$VAR3 = 3;
$VAR4 = 4;
$VAR5 = 5;
Perl> @a + 2
$VAR1 = 7;
Why adding a number to array results in number ? Because array in scalar context returns its length. It leads to some very compact code if (@a > 3) {print "big array"}
but, uh, what you do if you want to see length of string ? well length($string).Will that also work for arrays ? Nope, if you want to force scalar context you're supposed to do scalar(@array). So how to add 2 arrays ?
@c = (@a, @b) #
obviously. But wait, what we really typed after variable expansion is ((1,2),(2,3)) and in other languages irb(main):001:0> a = [1,2]
=> [1, 2]
irb(main):002:0> b=[2,3]
=> [2, 3]
irb(main):003:0> c = [a,b]
=> [[1, 2], [2, 3]]
>>> a = [1,2]
>>> b = [2,3]
>>> c = [a,b]
>>> c
[[1, 2], [2, 3]]
that's exactly what we get. Confusing ? Sure. But it saves few characters!The language models try, by statistical means, to derive what should be there. Given enough data they will start to have sone (statistical) grasp on the intention.
I am not entirely sure about the boilerplate though. Often you need some minor variation of an already existing pattern. Trying to unify those slightly divergent patterns into one schema can very easily lead to very hard to understand code. Another thing is thar boilerplate is fairly easy to write and to test, because it is familiar, reducing the actual effort which goes into it. Sometimes it is just better to not reuse code.
And then if it's buried deep enough somebody will add another layer and fix the cases that were found - there.
And that's how the disgusting legacy code happens.
KISS, please. Unnecessary abstraction is the root of almost all problems in programming.
Hell, just helpers/utilities functions within an organisation aren't always used, devs end up reimplementing stuff all the time simply because there's no easy way to know about it (documentation is only one part of this).
Although, I feel like very often, the idea of resucing boilerplate should __not__ to hide the boilerplate one layer below- it should be to __try not to write the boilerplate__ …
Take this very specific example at hand.. What is the meaning of this “JoelEvent” class? Why is it there?
It appears that is wraps a list of functions, with methods to push and pop from it. Why is it necessary to write them?
“Dispatch” reimplements function application , apparently? Btw, it is beyond me why one would loop over this.listeners and the check if the element is in this.listeners .
In a reasonable language or framework, I cannot in any way see a reason why this code needs to be written.. This is the idea of removing boilerplate, to me!
Sure, most code is boilerplate, except for that one thing, and that one thing can be anywhere. For example, let's say you want to write a function that returns the checksum of a bunch of data. That's a very common thing to do, there are plenty of libraries that do that, and I have seen the CRC32 lookup table in many places, sometimes I am the one who put it there.
Now, why rewrite such a function?
- Ignore some part of the message
- Use different constants
- Fetch data in a special way (i.e. not a file or memory buffer)
- Have some kind of a progress meter
- The library you may want to use is not available (can be for technical, legal or policy reasons)
- Some in-loop operation is needed (ex: byte swapping)
- Have a specific termination condition (ex: end-of-message marker)
- And many others, including combinations of the above
If you ignore all these points and only see the generic checksum function, yes, it is boilerplate and can be factorized. But these special cases are the reason why it may not be the case, and the reason why there are so many coding jobs.
It is also the reason why we don't have real (Lv5) self driving cars yet, why there are pilots in the cockpit, why MS Office and the like have so many "useless" features, why so many attempts to make software cleaner and simpler fail, etc...
As for legal or policy reasons, those still aren't reasons to write boilerplate code. Your reimplementation can be tight and reuse other abstractions or include their own.
In reality, few people need to write their own checksumming function, but sometimes, it is the best thing to do. And it is just an example, there are many other instance where an off the shelf solution is not appropriate because of some detail: string manipulation, parsing, data structures (especially the "intrusive" kind), etc... And since you are probably going to have several of these in your project, it will result in a lot of boilerplate. If it was so generic not to require boilerplate, it probably has been developed already and you would be working on something else.
Abstractions are almost invariably more complex, slower, more error-prone and generally worse than the direct equivalent. They are, however, reusable, that's the entire point. So one person goes through the pain of writing a nice library, and it makes life a little easier for the thousands of people who use it, generally, that's a win. But if you write an abstraction for a single use case, it is generally worse than boilerplate.
Any language with no boilerplate at all is a black box of incomprehensibility. Java has, I think, more boilerplate than average, while some other languages have less boilerplate than average.
IDEs can help with some of this, which is why I finally stopped writing all code in vim.
Yes, it's helpful that the system shows bugs, but it does this, not through careful analysis of the control flow or subtle type analysis, but by "probability of each token appearing given the previous tokens".
If such a large proportion of our code follows common patterns, are we not wasting huge amounts of time writing and testing the same functions across thousands or millions of pieces of code? If we (almost) always follow a certain pattern, should not that pattern be embedded in a library or language, so vastly reducing the opportunity for errors or bugs?