> You might say that surely that additional cost would be compensated in reducing the cost of maintenance later. Maybe but I'm not sure.
I am sure. 100%. From many years of experience.
Yes, static types come at an initial cost at initial development time. But they pay that time back in spades. This is exponentially true the larger the code base is (more to keep in one's head), the longer the project lives, and the more people are on it.
Having worked on very large C/C++, Scala and Python projects, when it comes to add a major feature or perform a serious refactor, I always want the static typing and the compiler to inform me when I've missed something. Far too many times has code been checked into a large Python code base that breaks something (completely unbeknownst to the programmer), because there's a code path that's rarely executed or arguments to a function flipped, etc.
That all said. There are major benefits to being able to prototype very quickly in a dynamically typed language, too.
You either have a type-checker and compiler, or you don't.
Duck types are great for writing new code, but they're very troublesome for refactoring; automated tools have a much harder time automating that process.
Refactoring data structures and implementations in static type systems is bother considerably easier to implement than in dynamic languages, and resultingly, more robust.
Certainly the refactoring tooling these days is pretty sophisticated with type inference, but... well, I've refactoring large python and javascript code bases, and my experience has been that absolutely a static type system makes that process easier, even if you have a comprehensive test suite.
I think it's worth acknowledging that there is a place for static type systems; certainly, it's not a silver bullet, and it results in (usually) a higher lines-of-code count, which is significant; but its naive and wrong to suggest that it has no value at all.
Specifically, for refactoring, it has a lot of value.
You can of course add explicit checks and tests, eventually paying the same amount (or more) in LOC as the typed implementation's initial cost, but then you're also tasked to keep those up to date, without the compiler's aid.
That layer is there anyway, except for your interpreter of a dynamic language only knows about problems at run-time, while the compiler that checks a static type system will tell you at compile-time.
A caveat is that I'm pretty sure my friend intentionally sacrificed code quality to do it, I don't think you'd find that project as readable and understandable as the others. Another caveat is that you have to be okay with your codebase being extremely magical and metaprogramming-heavy, which many industrial users of dynamic languages avoid.
As I mention, I'm personally into statically typed languages mostly for the performance and correctness benefits. I think it's plausible that on larger projects with teams the correctness benefits save enough debugging time that overall implementation time is lower, but I'm less confident in this now than I was before this comparison.
My hunch is however that what is often overlooked in development with statically typed languages is that it takes considerable time and effort to come up with the right set of types. Many examples are written showing how types almost magically make programs easier to understand. But when you read such an example what is not stated is how much effort it took to come up with just those types.
One way of thinking about it is that type-definitions are really a "second program" you must write. They check upon the primary program and validate it. But that means you must write that second program as well. It's like building an unsinkable ship with two hulls one inside the other. The quality will be great but it does cost more.
To be sure, having to write down those data structure invariants in a rigorous way that fits into the type system of your programming language has a cost. But the hard part really is coming up with the invariants, and it's dangerous to think that dynamic languages obviate the need for that.
A good example of this is matrix operations - there are plenty of invariants and contracts to check (e.g. multiplication must be between m x n and n x p matrices), but I don't believe there's yet a particularly convincing Haskell matrix library, in part because the range of relevant mathematical invariants don't cleanly fit into Haskell's type system.
For those cases, checking the invariants at runtime is your escape hatch to utilize the full expressive power of the language.
Surely. But if you have to write them down it becomes hard to change them because then you will have to rewrite them, and you may need to do that many times if your initial invariants are not the final correct ones.
The initial ones are likely not to be the final correct ones because as you say coming up with the invariants is ... the hard part.
What I'm trying to think about is that in a language that requires you to write the types down they have to be always written down correctly. So if you have to change the types you use or something about them you may have a lot of work to do because not only do you have to rewrite the types you will also have to rewrite all code that uses those types.
That does allow you to catch many errors but it can also mean a lot of extra work. The limitation is that types and executable code must always agree.
Whereas in a dynamic language you might have some parts of your program that would not even compile as such, if you used a compiler, but you don't care because you are currently focusing on another part of your program.
You want to test it fast to get fast feedback without having to make sure all parts of your program comply with the current version of your types.
A metaphor here could be something like trying to furnish a house trying out different color curtains in one room. In a statically typed language you could not see how they look and feel until all rooms have curtains of the same new color, until they all follow the same type type-constraints.
I've written once here before, this is one of the 'accidental advantages' of TypeScript: you set the compiler 'loose' when you're hacking away, writing quickly, and then 'make it more strict' as you start to consolidate your classes.
I almost don't bother to type something until I have to. Once I see it sitting there for a while, and I know it's not going to change much ... I make it a type.
It's an oddly liberating thing that I don't think was ever part of the objectives of the language, moreover, I can't think of any similar situation in other (at least mainstream) languages.
Add type hints at any time, check types at any time. Type hints can also serve purely as hints to programmers and not checked at all.
So it's the output of ONE Python programmer vs teams of other languages programmers?
It is well-known that teams cause overhead. Think of the Mythical Man-Moth.
But in the end the other teams had 3 people that were able to maintain and develop their project further, if needed. The single-person team had only one such person.
While writing the previous paragraph, I wrote 'programmer' where I had previously put 'implementation', because I would guess that this study's outcome is better explained by human factors than language differences.
I share the attitudes you state in your last paragraph, but I would add that we should be skeptical of concepts of quality that seem plausible, but which lack empirical evidence for their effectiveness.
That aside, speaking off the cuff totally separately from my post, I'm extremely dissatisfied with the performance of current compilers. The fastest compilers written by performance-oriented programmers can be way faster than ones you generally encounter. See luajit and Jonathan Blow's 50k+ loc/second compilers and the kind of things they do for performance. One example of a compiler task it's really difficult to do quickly in a language like Python/Ruby is non-regex parsing, I've written recursive descent parsers in Ruby and compiled languages and the Ruby ones were 1-2 orders of magnitude slower, non-JIT dynamic languages are really bad at tight loops like that.
Lua and Jai are lot less complex than say C++: sure, LLVM isn't necessarily built to be the fastest compiler in existence, but I don't think it's fair to compare it to compilers for much simpler languages and bemoan its relative slowness.
It's either that or assuming that your data will contain exactly what you want to, ignoring every possibility outside the happy path, which is a recipe for disaster.
That said, I have to say that modern JS produces the cleanest looking and less verbose code I've ever worked with so far. I wish there was a way to work with types without adding verbosity and harming readability.
> I think my overall takeaway is that design decisions make a much larger difference than the language
After all, Scala was 0.7x the size and is one of the most strongly statically typed languages. So you could almost invert your conclusion and say the big result is
"Python only saved 20% code lines over fully statically typed language"
The results say something about the greatness of Scala, not of statically typed languages in general. The other ones did not do quite as good as Scala.
From what little Scala I've read it looks very terse indeed. That can be a benefit but at the same time makes it harder to understand code written in it, in my opinion.
For example, it's not completely clear from the post but it seems like the 0.5x figure is from wc -l, which means Python wins a line every time there is a conditional or loop just because it doesn't need a closing brace. That alone might eat up a lot of the 20%, but you would be hard pressed to say that is a meaningful difference.
The reason I think this is "big big news" is I thought the general consensus had already been reached in academia if not the programming community that "statically typed functional languages are much better". There's little or no evidence of that in the results of this study.
If there's a surprise, it's that Scala is able to snuggle up to Python so nicely, while being one of most strongly typed languages.
More to the point: while the numbers nominally show differences in languages, everyone with programming experience recognizes that unmeasured and uncontrolled human factors probably had a big part in the outcome.
Scala has a very rich standard library. Python doesn't. An example is: https://stackoverflow.com/questions/363944/python-idiom-to-r...
Ruby would have been a better example of how succinct a dynamic-typed language can be.
(I made a similar comment on the parent level.)
This is not something I could have expected someone to say about Python's standard library…
Another example is that Scala offers a lot of ways to process a list like foldLeft, foldRight, unzip, headOption, lastOption, flatMap, groupBy, and many methods around Map, Set, and etc. Python probably doesn't offer many of these methods.
Of course, this comes with the cost of higher learning curve.
Actually, that seems to be the direction/principle of Python, where it is less inclined to add a helper function.
"It has been discussed ad nauseam on comp.lang.python. People seem to enjoy writing their own versions of flatten more than finding legitimate use cases that don't already have trivial solutions." from https://softwareengineering.stackexchange.com/questions/2542...
Not that this is better or worse. It's just that, on the brevity aspect, Python code is gonna be longer.
(1) It shows Python expresses the concept fairly compactly in the core language without even resorting to stdlib,
(2) It ignores an option (also in the core language) using a ternary and the truthiness of lists: my_list[0] if my_list else None
And we are talking about the brevity of a language here.
next(iter(your_list), None)
The degree of richness and/or the height of abstraction seem lower in Python. (Not that this is a bad thing. It depends on people's taste, of course.)
Like dragonwriter says, it shows that "Python expresses the concept fairly compactly in the core language without even resorting to stdlib".
It depends on your taste whether you like richer stdlib, and I do. But some don't.
We are talking about how short the code can be in this post, and `my_list[0] if my_list else None` (Python) is longer than `my_list.headOption` (Scala) or `my_list.first` (Ruby).
I'd appreciate more if you elaborate why my example isn't a good illustration on the brevity aspect.
The interesting takeaway from this study I think is that it does not show that statically typed (even "pure") functional languages are not obviously better than plain old Python.
The interesting thing is not what this study proves, but what it does not prove.
Could you elaborate more on this statement? Not a native english speaker here, so I don't quite understand the sentence. Thank you.
> A dynamic language like Python is better here, 2x better.
I was surprised by how small the LOC benefit was from using the dynamic languages. As someone who typically reaches for Python, I'd use a statically typed language (Go or Java, most likely) much more often if I expected only twice as many lines of code. In practice I feel the same project takes 3-10 times as many LOC and that pushes it to where it is more difficult to maintain and understand.
You have to be careful with that.
In a dynamic language your test system is your compiler. It is essentially a domain specific compiler. With a statically typed language, half your tests are already written, and you just have to activate them with a type signature.
For my money the trade off is down to whether forcing structures into a type system and getting the 'free advanced tests' is more advantageous than constructing a domain specific test harness 'compiler' and getting the flexibility of duck typing.
And you only learn that when you run up against the limits of the type system and have to start complicating structures to work around it.
In terms of a rough metric I'd suggest you have to include the test code in both cases to get a fair comparison.
> This does not say that static typing should not be used but I think it shows unequivocally that there is a considerable extra development cost associated with static typing.
Are you saying that the bottleneck in software development is typing in text on a keyboard?
If so, I thoroughly disagree. In my experience, inputting text is a small factor in development cost, with the main cost being research (figuring out what to type in), and debugging (figuring out why something you typed in doesn’t work as you thought it would).
I would agree, if and only if I thought a representative sample of Python programmers would all produce something of a similar size and just as correct, but I suspect, in this case, it's the result of one especially talented person.
I like that the title frames it as a language shootout to pull people in to see if their favorite language wins (and I'm partial to Python having rewritten tens of thousands of lines of Java into numpy). Still, it would be foolish for people to come away from this brilliant analysis by ignoring the more important conclusion.
Having said that, the largest point I took away was that the difference between languages was smaller than the difference between programmers and approaches.
This can be exemplified using Crystal, which is close to Ruby in both terseness and APIs, but statically typed (and with static dispatch).
A dynamic language like Python is better here, 2x better. I assume similar results would apply to other dynamic languages like JavaScript, Lisp, Smalltalk, Groovy etc.
This does not say that static typing should not be used but I think it shows unequivocally that there is a considerable extra development cost associated with static typing.
You might say that surely that additional cost would be compensated in reducing the cost of maintenance later. Maybe but I'm not sure.
Any development effort of significant size (like writing a compiler here) is a combination of writing new code and adapting and modifying code already written. "Maintenance" is part of development.
This is quite a quantitative study which gives credence to the claims of advocates of dynamic languages.