I inevitably came the exact opposite conclusion. Perhaps because at the start of my career I was trying to do the same thing with Java, and it added overhead and expense without a commensurate payoff in safety.
Frankly, I think velocity is ridiculously underrated:
* Because of the age old "customer doesn't know what they want" issue - hence you throw away a lot of the code that was not fit for purpose.
* Also because the right kind of architecture is only obvious in retrospect - hence you throw away a lot of the code that was not fit for purpose.
* And finally, because the only times in my career where I've had significant success in refactoring a large mission critical codebase it was python and it was because I'd managed to quickly build custom tooling infrastructure (e.g. testing) to support that refactoring.
I do take some issue with python's type system (e.g. the whole truthiness thing), but on the whole it balances safety and velocity in a highly pragmatic manner.
Java appears to be the most bureaucratic language around where most code is written by people who are just a bit too enthusiastic about OO so every class becomes a FactoryManagerPatternVisitorImplementation and every struct 'of course' needs getters and setters.
Python on the other hand is often written without any safety and I have frequently witnessed how every typo becomes a new variable maybe just because the cat stepped on the backspace key and how one developer changes the signature of a function but not all places where it is called.
I think there are great benefits to compile/lint time type checking for anything bigger than just a throwaway script. For python one can use mypy, which helps. There is no need to, though, to turn everything into a festival of object orientedness. Most classes need neither parent nor child. And some functions can just be global functions.
>Python on the other hand is often written without any safety
Oh for sure, but I find it easy to start dialing up python's safety from day one of starting on a project (e.g. start writing asserts), no matter how complicated.
Working around java's horrible type system? Less easy.
If you want something closer to the extreme, go for Haskell. But then, you can't complain much about bureaucracy.
There is an annoying tendency of people to base their opinions on technology directly copied from the 70's.
It really sounds like your code base simply wasn't too big after all. When you change a Java API, how much time does it cost you to find all broken client code? How much in the same case for python?
> Frankly, I think velocity is ridiculously underrated
Again, it sounds like your projects were still rather young. IME, throwing away code practically never happens when it has been installed for at least half a dozen customers. So python helps you to build stuff quicker, because it does not force you to be sound. But soundness is exactly what you want after a certain threshold.
With a decent test suite and a bunch of well crafted asserts, it ends up being much quicker. Indeed, for the really subtle broken stuff, pretty much only a decent test suite can catch it.
>Again, it sounds like your projects were still rather young.
Probably just better tested than yours.
In time, you might learn to appreciate this quality but who knows...
I think you don't understand the point. Unless you propose that my test suite tests the libraries I am using, testing does not find code affected by API breakage. And even if you do test your used libraries in your client code, think again about how much effort it cost you to do what a machine can do just better.
> Probably just better tested than yours.
First of all, I doubt it. Second, you just sound rather arrogant.
I hope you are aware that a) testing by necessity will always be incomplete and b) type systems by necessity will always be restricted.
The developer who just discards one of these tools sounds like a woodworker who discards a chainsaw because he loves his splitting axe so much.
Oh, the irony.
"It really sounds like your code base simply wasn't too big after all"
"Again, it sounds like your projects were still rather young"
My point was that you accuse the parent of arrogance, while he was merely mirroring/responding to the arrogant tone in your comment (in the lines I've quoted).
That is, my point wasn't that you made an absolute claim ("your code is X") that was false. If that was my claim, yes, you could refute it by saying "I only said that it sounds like X is the case, I didn't make an absolute claim".
But my point was rather that the tone in your comment was arrogant.
"Your code base simply wasn't too big after all" is equally arrogant whether prefixed with "sounds like" or not.
It makes assumptions about the size of the other person's codebases, and it arrogantly implies that they don't see what they see because they haven't worked in as large codebases as you have.
I do find that well crafted integration tests give the best bang for the buck.
>First of all, I doubt it. Second, you just sound rather arrogant.
This was a mirror image response to your condescending assumption that I "just hadn't worked on large enough code bases"
It's a little sad that you didn't have the self awareness to recognize that my feigned arrogance was satirizing yours ;-)
if you really worked on large projects you would know that integration tests involve more than just one language. They are a helpful tool, and a very important one. But if I had to rely on integration tests to figure out API breakage, I would have to quote a massive price for any libray upgrade. Fortunately I have cheaper and better means.
> It's a little sad that you didn't have the self awareness to recognize that my feigned arrogance was satirizing yours ;-)
Yeah right. You quote two projects and consider your work better tested than mine. And I am the arrogant one. All I said is that you sound like you did not see the kind of projects I did.
Here's the thing. You either are really up to something and know something that all us soundness-first old farts doni know. Then you are going to revolutionize software development that has been coping with the same issues since the 70s. Or you are simply an arrogant youngster full of Kool aid, and you will crash against the same walls everyone else in the industry did.
And I don't see how your 3 points about the velocity thing are affected by static types.. Like, come on, many points can be argued over compiler usefulness, but help with throwing away code is rather not one of them.
Because the OP was about python and because python and java are the two languages where I have worked on refactorings of large scale code bases.
I can't help but feel that if you make a hypothesis (e.g. "static typing is naturally better"), you need to be open to examples that disprove this hypothesis.
>And I don't see how your 3 points about the velocity thing are affected by static types..
Static typing usually means a lot of time spent satisfying the compiler. I don't disagree that this can be useful because it catches bugs (although a lot of stuff caught by compilers are not bugs). I disagree that this isn't a cost/benefit trade off that naturally works out.
> I disagree that this isn't a cost/benefit trade off that naturally works out.
I agree, it is. I just think that programming languages are a whole package and Java is really bad with this cost/benefit ratio. I'd rather see something else being an ambassador of static types.
Types are an intrinsic part of ergonomics.
>I'd rather see something else being an ambassador of static types.
I'd be happy to see another language be an "ambassador of static types" provided the tip of the iceberg of 'visible things built in it' doesn't compromise of say, one spam filter and a file converter.
I'm getting pretty tired of hearing about how awesome and productive haskell's type system is for 15 whole years and seeing a whole lot of evidence that it's waaaaaay too unproductive to build anything of use in it.
Rust is a fine ambassador of static types (and has some great usage examples), but rust and python address extremely different non-overlapping markets and rust isn't shy about the fact it makes it fucking hard to compile things in.
For example creating new type in Java and Rust is a whole different world. If you can't separate Java the language from the idea of static typing and type systems, you are probably not a person suited for such discussion.
As for your remarks about Haskell, welp, I have build things with Haskell and Python, and it's not even close. But I guess chasing runtime exceptions during development is an acquired taste.
I can. It's people who make blanket statements about static or dynamic typing who can't.
>I guess chasing runtime exceptions during development is an acquired taste.
Much like building software people actually use.
> When you see a huge codebase in python grown over several years and try to refactor it, reason about it, or just enhance it, you will inevitably come to the conclusion that the omission of static type checking is a grave error from a system-design perspective.
I never did.
When you give a C++ or Java programmer a large Python codebase, they'll find it hard to work with, and conclude that the problem with Python is the "omission of static type checking", i.e., the defining feature that their previous language had.
Coming from Lisp, when I found large Python codebases hard to work with, I concluded that the features of Lisp (macros, symbols, conditions, multimethods, dynamic vars, feature-complete lambdas, etc.) were the problem. Static type checking is one tool to help deal with the massive quantity of dirt [1], but better abstraction capabilities mean there's an order of magnitude less dirt to deal with in the first place.
Yes, static types are a means of communication. But they really only communicate what's already there. Conditions and multimethods are also means of communications, and they enable me to say what I mean in higher level terms instead of spelling it out in lower-level constructs each time. I don't worry about "refactoring tools" in Lisp because I don't have to repeat my intent all over the codebase.
Most people come from programming languages with static type systems. The most popular dynamic languages in 2019 (Python, Javascript), unfortunately, lack the great abstraction-building features that Lisp has. It's not surprising most people extrapolate from their Py/JS experience into believing the problem with dynamic languages in general is that they don't have a static type system.
[1]: http://steve-yegge.blogspot.com/2007/12/codes-worst-enemy.ht...
A type tells me what it is in any version of the program that the compiler accepts.
It doesn't tell me their significance for the business logic, that's what variable names are for.
If you are making a linear collection, backed by a resizeable array, your interface is clear.
If you are making something like an I/O subsystem of an OS, in 10 years your primitives suddenly may change. For example, flash storage replaces spinning disks, and you need to schedule block erasure preferably when user programs are idle. Now you need to find all the places blocks are detached from the file structure, excluding FS journaling, and change the logic there to accommodate new scenarios, trying to reuse as much code as possible for you everyday operations. In a statically typed language I can perform exhaustive search for all such cases in no time, and make guaranteed safe refactorings to deal with that problem. No so in dynamic languages.
The change happens even faster with 3rd party networked services/libraries, because their evolution is not held back by expensive experimentation.
I'm not saying it's a bad idea, just that it's not really a core application of type systems in real-world programming practice.
You don't need too advanced of a type system to implement what you're talking about. With the caveat that doing it in Java would make the code tedious to write ;-).
I like static type checking for what it does, but it solves some little problems, not big ones. (And can cause their own little problems.)
Inability to do automatic, reliable refactoring is a big problem.
Inability to detect a wide range of errors before hitting them at runtime, big problem.
There are many more.
Mere mortals can’t construct type systems to enforce most definitions of correctness or licenses or performance.
That's not true generally. Dependent types can be used to represent any property that a test might check, and lightweight alternatives can prove a subset [1]. And this makes sense from the type of logical property being established: you can easily turn a universally quantified proposition into an existentially quantification, but going the other way around means checking all possible permutations of values.
However, proving a universally quantification is sometimes (often?) far more difficult than proving an existential quantification, which is why programs have to be structured in a certain way to make them amenable to such proofs.
> Mere mortals can’t construct type systems to enforce most definitions of correctness or licenses or performance.
Type systems always improve performance, so I don't know what you mean by that exactly.
For programming in the large, both should be used. In the long run it’s about saving time that is otherwise wasted on wolf-fencing and hair-pulling; and in getting new developers up to speed faster.
Consider yourself lucky.
Your stated "big problems":
* developers often aren’t sure what it does
It's easier to understand what code is doing in a strongly typed system. Just trying to find where a method is defined in a dynamic language is often difficult and time-consuming.
* how to organize the code
Strongly typed languages generally encourage better code modularity. Not a panacea, but a help.
* aren’t designing for maintainability
Automatic, safe refactoring is a crucial tool in the maintainability toolkit. Again, not a panacea, but a huge help.
I understand there are always bigger problems, and adopting a strongly typed approach after a mountain of technical debt has already accumulated may not be worth it. And if your developers can't write good, maintainable code regardless of static/dynamic types, that is certainly a bigger problem. But none of this means that static types don't solve big problems, which was the assertion I disagreed with.
But I think their criticism comes from an ivory tower where people in general agree that efficient programming needs strong and sound tools.
Unfortunately, in practice it seems to be all about "velocity". When you see a huge codebase in python grown over several years and try to refactor it, reason about it, or just enhance it, you will inevitably come to the conclusion that the omission of static type checking is a grave error from a system-design perspective. And when you work with the developers of such a code base you will understand that static type checking is first and foremost a means of communication between the language implementation and the developers. Without this tool it is like taking away a common spoken language between the developers and having to translate everything into some form of pidgin.