The Python version is around 1/3-1/2 the size of the Go version, but the clarity with which bugs are visible in the Go version is astonishing.
There's a lot of code in Python version that "looks right" and even "does kind of the right thing a lot of the time." But the Go version lays bare corner and edge cases that were not obvious in the concise Python version.
I agree that Go usage does suffer sometimes from lack of generics. I've sometimes used code generation in its place, and it works, but it's a poor solution. At the same time, the lack of generics has helped me avoid over-abstracting in other places, which is a common bane of mine whenever I work in more "expressive" languages.
Besides, in practice I find lines of code in Go vs. Python are only maybe 20% different in many common cases, if you ignore the struct declarations which I have a hard time calling "code". (Being not a Turing complete language, that particular subset can't be considered vulnerable to the issues that Turing complete things can have.) I find myself very frequently wondering how many people complaining about all the lack of abstraction in Go have A: used it and B: used it to the point they're writing Go in Go. (Go just blows if you're trying to write Python in Go or something, or if you insist that you know better and must use map, filter, etc. rather than for loops. For all languages X, writing X in X is important, but Go is really graceless if you don't learn how.)
Now, if you do get to B, it is still true that Go does not quite abstract as "well" as some other languages, but it does abstract well enough that you can start having reasonably conversations about whether those other abstraction tools are actually paying their way in a lot of larger, shared code bases. If we throw out a ton of language complexity, but only pay a relatively small price, are they really carrying their weight? The answer will vary from code base to code base, but I think it's not necessarily some sort of horrid, unthinkably idea that not every program must be assaulted with the most powerful, complicated toolkit regardless of the price incurred by the complication.
The results in those "past decades" used all kinds of languages, some far more advanced and modern than Go is today (e.g. Lisp, Smalltalk, Ada).
In a sense, Lisp is the most advanced language ever designed, and still, it is ignored by most of the programming community :). Equally, I think, Smalltalk does object orientation better than most languages which followed it.
Both languages are mostly dynamically typed, so comparisons to static typed languages need to take this into respect, they are different beasts. On the other side, I think Go has an interesting mix out of static and dynamic typed features, having interfaces for static contracts and interface{} as a fully dynamic type, which still offers full type safety at runtime.
Go structs have also some interesting properties, if you look at them beyond being containers. Any type can have methods, and structs can embed other structs and thus "inhert" their methods in a sense. This is not full blown object orientation, but gets you surprisingly far.
For example when should I use NamedTuple and when should I use a dataclass? Should I be using a list comprehension or a map/filter/reduce? When should I split that list comprehension? I have seen (and made myself) terribly long lines with only list comprehension and it was hideous.
yes, use list comprehensions -- they are more readable.
>When should I split that list comprehension?
just use common sense. if you can read it and understand without thinking too much, that's good.
it's readability beats short code 100% of the time.
It illustrates what you have to pay to get this kind of functionality for arbitrary slices.
I wonder how this would change with generics.
That said, anecdotally I too have found it much easier to find errors visually in go programs due to its simplicity. I find the same is true of C vs. C++.
This is (or was) deliberate, with the rationale being that it makes O(n) blocks of code trivial to identify in review.
Whether or not you buy that argument is a separate thing, of course.
How is such identification made easier by manually writing out a `for` loop applying a function foo to each element rather than writing `map foo myCollection`?
Memory allocation is so expensive that making that copy is often more expensive that calling `foo` on each element.
Sometimes making a copy is exactly what you need and there's no way around that cost (but hold that thought).
But I've also seen `sum map foo myCollection` so many times (especially in JavaScript).
Here you have a short, neat but also extremely wasteful way of doing things. I see it so frequently that I assume that many people are unaware of this cost (or completely disregard performance concerns).
If you were to write this imperatively, it would be obvious that you're making a copy and maybe you would stop and re-think your approach.
But there's more.
If you're paying attention to performance, an easy way to improve performance of `map foo myCollection` in e.g. Go is to pre-allocate the resulting array.
For large arrays, this avoid re-allocating of underlying memory over and over again, which is the best a `map` can do.
In imperative code those costs are more visible. When you have to type that code that does memory allocation, you suddenly realize that the equivalent of `map` is extremely wasteful.
Your points about efficiency are a separate topic entirely from the original claim that manually writing out an imperative solution makes it easier to see the algorithmic complexity. That was a surprising claim to me because, in my experience, if I understand what some HOF is doing, reading the code is even easier because there is less of it to wade through (and mental exhaustion doesn't make one easier to read vs the other).
> In every mainstream language I can think of `map foo myCollection` creates an intermediary map
You need to build up the final result, sure. Not an intermediary map but whatever structure (or functor) you're mapping over. That is the whole point of immutable data structures. Also when you're using persistent data structures, which all modern FP languages do, the cost of constructing the result can be far less than what you expect, especially if the result is lazy and you need only some of the results. There is a cost to immutability and if it's unbearable in some situation, fall back to in-place mutation but the semantics of these two approaches are definitely not the same.
> But I've also seen `sum map foo myCollection` so many times
Yeah... that should be a fold (reduce, whatever). :)
Language support for Transducers can fix this, you can compose functions like map / filter / reduce over a collection and only hit each item once.
numbers.iter().map(|n| { ... }).sum();
Compiles to a plain loop, with no allocation.Would anyone expect such a developer to read and parse several more lines of code more reliably, in order to understand the algorithmic complexity? Seems unwarranted to me...
In programming, if you miss an important detail the repercussions can be high. In a million line codebase every idiom that's slightly more complex than it needs to be is going to result in dozens of additional bugs, sheerly because of the increased surface area for making mistakes with that idiom.
I think the clue to your thinking, for me, is in your description of the `map` HOF as "slightly more complex". Having years of experience with both paradigms, I've found that grokking a call to one of these fundamental building blocks (map, filter, reduce, fold, etc) is nearly instantaneous. We've all experienced reading prose where the author was excessively verbose when the same point could have been made succinctly. It feels the same way reading for loops once you get over the learning curve of these very basic functional constructs. You have to keep repeating that boilerplate endlessly and it's very tedious to keep writing and reading it.
Yeah, and it's much easier to make mistake with a loop than with a map or filter.
Whereas, greping for .map(), .filter(), .reduce() wouldn't?
Like how
for (int i = 0; i < strlen(str); i++) { ... }
is O(n)? for (int i = 0; i < strlen(str); i++) { ... }
could be grepped and easily discerned from: for (int i = 0; i < str_length; i++) { ... }
and generally, that every looping point could be easily identified.After all, if you call functions inside the body, all bets are off as to what complexity the loop means anyway...
If the lack of expressiveness equals to greater amount of code and thus more bugs (in Go), does the same apply for languages that are more verbose (say, Java)? And if language is more expressive and less verbose (say, Python or Javascript maybe?), does it mean less bugs?
The number of bugs in a program correlates with the number of LoCs, but that's about it (correlation does not imply causation).
The claim is also quite absurd, since, for example, you can pre-process any C or C++ program ever written into a single line of code, yet this operation doesn't reduce the number of bugs in these programs, therefore the amount of bugs is not "just" a function of the amount of LoC.
Languages with better abstraction capabilities than Go (e.g. Rust or Python), might require programmers to type less, but the resulting lines of code are often dense. For example, I'm quite comfortable with Rust iterators and Python lists comprehensions, but it still takes me much longer to parse what code using these abstraction does, than code that just uses a simple for loop. It definitely feels great to cram a 10 LoC loop into a 1 LoC list comprehension in python, but reading that code 6 months later never felt that great, even for code that I've written myself. I just need to stop at those lines for much longer.
for (dd=d[k=0]; k<16; dd=d[++k])
{
satd += (dd < 0 ? -dd : dd);
}
The problem with for loops is that they are too flexible, and so there's more opportunity for misuse. Iterator functions are less flexible, and therefore easier to read.And this excessive flexibility doesn't even make simple things simple. Try counting down from N to 0 inclusive using unsigned ints.
I'd like to see how solving this same problem in those languages is less of a nightmare.
I personally find that C loop quite readable compared with the same iteration being done in Rust.
let satd = d.iter().cloned().map(abs).sum();
As an added bonus, there's no undefined behavior! for k, dd = 0, d[0]; k < 16; k, dd = k + 1, d[k + 1] {
...
}
I don't find that significantly more readable than the C version.No, but it suggests it, and in this case, there's not many other options that could explain this than causation.
>The claim is also quite absurd, since, for example, you can pre-process any C or C++ program ever written into a single line of code, yet this operation doesn't reduce the number of bugs in these programs, therefore the amount of bugs is not "just" a function of the amount of LoC
That's an irrelevant strawman. When people speak of LoC they mean as to how many LoC are needed for various tasks based on the expressiveness of the language. Not whatever old LoC based on arbitrary adding needless lines (e.g. by preprocessing to inline function calls)...
But when looking at OP:
> bugs in software has been shown to be a function of LoCs [1,2,3].
He provided three different sources! Maybe it's wrong, or maybe you don't believe it, maybe it doesn't fit with your beliefs, but you can't say the OP claim is unsubstantiated when he provided academic sources on the subject!
This is an interesting study on programming languages' effect on software quality: https://web.cs.ucdavis.edu/~filkov/papers/lang_github.pdf
I'm not sure it makes sense to consider code correctness separately from productivity. You can probably get code written in any language to roughly similar levels of correctness. But how much work does it take to get there? That's what I'm really interested in.
One example of Go's lack of expressiveness is that loops are not abstracted away, e.g. you can't use higher level constructs like map and filter on collections. There are other languages that IMO strikes a better balance between simplicity and expressiveness, like Python and Elixir.
1. https://ieeexplore.ieee.org/document/1702967
2. https://link.springer.com/chapter/10.1007/978-1-4471-3003-1_...
3. https://www.amazon.com/gp/product/0735619670/ (Chapter "Developer Testing")