> Python relatively recently added typing[0]
The typescript compiler and the mypy type checker both appeared in 2012.
Function annotations have also been officially part of the Python syntax since 2006, although they could be used for more than typing.
Browsers still do not support the typescript syntax as of today, you need a transpiler.
> parallelism support (recently in v11 w/ worker threads), and built in async IO paradigms give Node a huge leg up over ruby, python, and perl.
Python has supported non blocking processing as far as 21 years ago with threads. It's actually the language of choice for pentesting a networks because it's so easy to implement a custom service scan.
Python then, in 2002, saw the rise of Twisted for pure asynchronous I/O + callbacks, as it was for JS and AJAX at the time. Ansyncio came way later, yet Python implemented async/await and yield before it was part of JS.
Python implemented multiprocessing as well, which allows to leverage multiple cores, more than 10 years ago. I remember that producing the equivalent result ishard to use accross all major browsers, and nodejs, consistently. The Python multiprocessing pool has even not changed API when we migrated from Python 2 to 3 and works with both (I just tried it to be sure).
> Weirdly enough, JS is the best in my opinion precisely because it does so much to not be itself.
I think you should try to put in production 4 or 5 other languages before stating such a strong opinion. Your post feels more like over enthousiasm due to lack of experience than a real analysis of the state of dynamic programming languages.
There are strong qualities to Perl, PHP, Python and Ruby. The fact they are so popular without having an accidental absolute monopoly on the most extraordinary plateform of the world (the Web) should at least give you a hint of that.
> Function annotations have also been officially part of the Python syntax since 2006, although they could be used for more than typing.
> Browsers still do not support the typescript syntax as of today, you need a transpiler.
I did not make a statement about the typescript compiler/transpiler nor mypy, I made a statement about the typing module which was brought about by PEP 484 (proposed in 2014) and accepted in 2015 a full 3 years after the typescript compiler came on the scene. JS got gradual typing first, even if it was through transpilation -- there are even earlier examples of syntax enhancement (though it was mostly sugar) with coffee script from even earlier.
JS is undeniably more innovative in this aspect -- it's also made more mistakes as well.
It doesn't matter what browsers support, because the code is transpiled. The point is that you have access to better tools as a developer while writing code.
Is your point that python have been specifying their types in their code in any sort of numbers since mypy was invented? Because I sure don't see that on slides PyCons over the last ~5 years.
> Python has supported non blocking processing as far as 21 years ago with threads. It's actually the language of choice for pentesting a networks because it's so easy to implement a custom service scan.
> Python then, in 2002, saw the rise of Twisted for pure asynchronous I/O + callbacks, as it was for JS and AJAX at the time. Ansyncio came way later, yet Python implemented async/await and yield before it was part of JS.
There's a difference between what's possible and what's easy/facilitated well by the standard library of a language. I did not imply that python could not do non-blocking processing. My point was that these patterns were not easy and were not standard -- and the community floundered for years between alternate solutions (gevent/twisted/tornado), eventually creating asyncio while NodeJS has had support for this feature built in from the start.
Also I'm not sure that pentesting stat is correct -- ruby is extremely popular and so is/was perl. Scripting languages with decent standard libraries and easy semantics are popular, not necessarily non-blocking processing support.
> Python implemented multiprocessing as well, which allows to leverage multiple cores, more than 10 years ago. I believe producing the equivalent result is still hard to use accross all major browsers, and nodejs, consistently. The Python multiprocessing pool has even not changed API when we migrated from Python 2 to 3 and works with both (I just tried it).
Please see the other comments. No one is saying multi processing is impossible in python, I specifically called out that it's done by starting other processes. There are subtle differences between how node worker threads and multi-process setups work, in particular the shared memory space and communication mechanisms. The GIL is also an issue.
> I think you should try to put in production 4 or 5 other languages before stating such a strong opinion. Your post feels more like over enthousiasm due to lack of experience than a real analysis of the state of dynamic programming languages.
> There are strong qualities to Perl, PHP, Python and Ruby. The fact they are so popular without having an accidental absolute monopoly on the most extraordinary plateform of the world (the Web) should at least give you a hint of that.
So your point is that everything is different and everything has it's own benefits? Well that's impossible to refute so I'm just going to restate my point, maybe you can try and refute it directly.
Javascript has the accessible (and thus widely adopted) and easy to use implementation of the following features, today in 2019:
- Rigorous and powerful compile time type checking
- Parallel computation (without spawning another process or the effects of a GIL)
- Asynchronous IO
The combination of these three features being accessible is what makes Node stand out in front of Perl, PHP, Python, and Ruby.
Python may have "had" the capability to do the things above at various times in the past, but it's technically been capable of doing just about anything, like any other programming language -- you can launch sub-processes from bash, doesn't mean you should use it for large applications. What makes a good language is accessible and easy to use features that make you both correct and productive.
Node has more of these tools available, and they're more accessible in Node than they are in Python, and more people are using them. Typescript offers more robust type support than typing+mypy. Node worker threads, though very recent, provide parallel computation without child subprocesses, in a shared memory context with no GIL. Asynchronous IO is second nature to JS developers, while that's definitely not the case for python developers.
> JS is undeniably more innovative in this aspect -- it's also made more mistakes as well.
JS has none of that. Typescript has it. They are not the same thing.
I'm comparing an external tool with an external tool. Mypy could type check Python before it was officially in the stdlib, just like the typescript type checker can make it so. Type annotations are not part of JS.
> It doesn't matter what browsers support, because the code is transpiled. The point is that you have access to better tools as a developer while writing code.
It does very much. Transpiling is not a free operation, and comes with a lot of pros and cons.
It's also not a universal operation. Most JS projects do not use typescript. Many projects still don't even use a transpiler at all.
> There's a difference between what's possible and what's easy/facilitated well by the standard library of a language. I did not imply that python could not do non-blocking processing. My point was that these patterns were not easy and were not standard
JS async were not easy neither at the begining. We had a decade of callback hell, then another war about the promise implementation, all that wrapped up in code to make browser differences manageable. Not to mention the 2 nodejs forks in it's small lifetype, JXcore rewritting the entire async model.
Let's also not forget we managed to scale web services way before NodeJS was a thing, with dozen of languages.
Yes, making an async HTTP request is more natural in JS than in Python. It would be a shame, JS is a Web oriented language while Python is a polyvalent language. However, I have yet to witness a service where this would have made such a huge difference all in all. Even facebook use long polling if I recall, something you can do with threads easily on the client side, and support with a proxy such as nginx as a front end even with nodejs as a backend.
> Javascript has the accessible (and thus widely adopted) and easy to use implementation of the following features, today in 2019:
> - Rigorous and powerful compile time type checking
Not JS. Typescript. And typescript and mypy came out together.
> - Parallel computation (without spawning another process or the effects of a GIL)
Web worker consume a similar order of magniture of resources than a python instance, and communicate data by message passing the same way. And no GIL.
The difference ? Python came fist, and has a more consistent support (you need fallback in JS if you use it in a website). I don't think we should be proud of it, but it's not the middle age you are talking about.
> - Asynchronous IO
> The combination of these three features being accessible is what makes Node stand out in front of Perl, PHP, Python, and Ruby.
The advantage of JS is historically a better async API. Let's not overstate this though.
And even if JS were vastly superior in all those area (which is, again, overstated), it hardly is enough to draw the conclusion that:
> as the best feature set of the "scripting language" tier of interpreted programming languages
Readability, ease of automation, rich stdlib, ecosystem, ability to deal with formatting/parsing/date handling, community culture and so much more are as important, if not more important, for a scripting language. Scripting, after all, goes far beyond the Web.
> Python may have "had" the capability to do the things above at various times in the past, but it's technically been capable of doing just about anything, like any other programming language -- you can launch sub-processes from bash, doesn't mean you should use it for large applications. What makes a good language is accessible and easy to use features that make you both correct and productive.
Youtube having been rewritten in Python proves that it was more than being capable. It also powers instagram and reddit.
When google rewrites their Python services for perf, they don't go for JS, they choose Java, C or Go. That's because again, Python is at the same order of magnitude of perf than JS on average.
> Node has more of these tools available, and they're more accessible in Node than they are in Python, and more people are using them.
Yes, JS is a Web language. That doesn't make it the ultimate scripting language.
> Typescript offers more robust type support than typing+mypy.
This is a gratuitious claim. Espacially since typing is in the stdlib, while typscript annotation are a syntax error in Emascript.
> Node worker threads, though very recent, provide parallel computation without child subprocesses, in a shared memory context with no GIL.
Objects passed to web workers are serialized and deserialized, just like with python multiprocessing.
> Asynchronous IO is second nature to JS developers, while that's definitely not the case for python developers.
Doing Ajax call being the based of today's web, the contrary would be alarming.
You could say the same for languages that require compilation, it's not exactly an uncommon pattern. JS will almost always have some kind of operation applied to it when running on the web (minification etc) so adding TypeScript transpilation is often a very small step.
Is that ideal? No. But Python doesn't even run on the web, so of course it doesn't have these issues.
I'm not high-confidence about this heuristic because 1) the "class label" is a bit subjective and 2) I don't have a massive sample size, but attitudes toward typing is one of the best filters for engineer quality I've come across.
(No, I haven't put this in practice in interviews, and I don't ask any related question. It's just a correlation whose strength I've found really striking)
It's part of a broader class of heuristic I've been noticing recently (again, still a very preliminary, low-confidence model): things like "lack of reliable, quality devtools" and "inability to customize interfaces" bother highly-productive engineers more than less-productive ones, which is what you'd expect given that inefficient interfaces to their work has a far greater cost to the former. Along these same lines, if (eg) you're already pretty bad at understanding complex systems by reading code, you're much less likely to notice or care about how much more difficult dynamic typing makes this.
I should be clear: this isn't a general-purpose argument against dynamic typing, and I'm not suggesting that the situation is as simple as "dynamic typing bad, static good". They both have their strengths and weaknesses, which make each appropriate for different situations; I ended up picking Python for a company whose tech org I was responsible for building, and as much as I personally find engineering in Python to be horribly unpleasant, it was the right choice for the company given the environment it was in, and I would make the same choice in the same situation.
I don't think my theory above is particularly novel, and I understand the self-serving bias that it aligns with, which is part of why it's still such a low-confidence model. But I've been unable to wrap my head around the number of nonsensical defenses of dynamic typing I've seen over the last few years, and this model is the first to offer an explanation: if you can't tap into the benefits of static typing, of course the accessibility and flexibility of dynamic typing is going to seem like it far outweighs the foofy, theoretical benefits that static typing brings to stability and comprehensibility.
I think for me it is the other way around actually: I believe I'm worse than the average programmer at keeping complex states in my head, thus I push heavily for high quality code and tools to reduce the times I have to fight through particularly nasty debugging / refactoring sessions .
No one ever talks about it, but one of the most valuable things I've learned to start doing is writing pre-push (and sometimes pre-commit) hooks into my projects, adding the corresponding `dev-setup` or whatever command to put them in place, and running linting & unit/integration tests before every commit/push. You could solve the problem of committing/pushing buggy code by just "writing better code", but with this methodology that problem completely evaporates.
If you think you are actually good at keeping complex states in your head (not just better than the average human/programmer) and therefore don't need high-quality code/tools, you are setting yourself up to be destroyed by your own hubris. There are no humans who are good enough at keeping complex states in their head to actually understand the gigantic programs that we work on these days. Back when a few hundred kb of memory was a lot, some people might have managed it, but not anymore.
You're moving more quickly and at higher levels of abstraction, so getting dragged down into the mud of reading every line of some shitty code to figure out an ambiguous type or side effect has a far greater cost (the lack of const or anything similar makes it even worse, since you can't write off a function that takes an object even if your sphere or concern is just that object). By contrast, if the functions you're writing are trivial and you read and understand code line by line, dropping into the stack just blends into the rest of the code-reading (or so I assume). (I still remember inlining an absolutely grotesque type-hint in order to get some ammunition to convince someone to refactor his sewage spill of a function. I wish I had written it down, but the type hint had something like twelve tokens and three levels of nesting at it's deepest point. Even that guy wouldn't have written that code if he had had to write out the type during development, or at least I like to think that. Shudder)
The rabbit hole goes very, very deep when it comes to engineering talent (or lack thereof...). You're probably a much higher-percentile quality engineer than you think, as I've met people working in industry whom[1] I had to fight tooth and nail to institute the absolute basics of common-sense engineering practices. The fact that you're aware of the value of static typing is a pretty good signal IME, and as I said, hoping that typing et al will "save you from yourself" just means you don't have the brain of a computer, and isn't a habit that talented engineers lack.
[1] the long and short of it is that I took an ill-advised shot at being the first employee of a company with two non-technical co-founders; I wanted something very specific out of the opportunity so it was fine for me (or so I thought...combining founder psychology with two non-technical founders is a freaking nightmare), but I wasn't willing/able to tap people in my network, all of whom were at much better organizations doing much more interesting things and getting paid tons more. So I got a nice look at the underbelly of the engineering labor market, enough to make me stay far, far away from small startups unless I have a very, very good reason to want to work with them or very good assurances that they'll be able to hire at at least the baseline level of a big company.
Typing out code tends to be a minor part of writing software, so you don't gain much there. Furthermore, you don't want to mix types in the same variables either, because this comes at a cost of performance and complexity.
So, what kind of flexibility do you get from dynamic typing?
It's got a lot better in the last decade, but with the more advanced features of a language you can find yourself slowed down in a few ways. Maybe by having to explicitly declare types where you'd normally implicitly declare them. Or you might have a load of methods that actually throw 'Not Implemented' errors because you are forced to match a massive interface but it only has to do one thing (yes, C# has an actual NotImplemented exception because this is so common).
It's annoying and can interrupt your flow, mean you have to spend a bit of time searching for the right voodoo code to make it work or you sometimes feel like you're doing a rain dance just because of the type system.
The other thing is that you're sometimes forced to cast a variable to a type that you know is already that type, but a method has returned it as an 'object'. It feels really clunky, e.g. event listeners.
> Or you might have a load of methods that actually throw 'Not Implemented' errors because you are forced to match a massive interface but it only has to do one thing (yes, C# has an actual NotImplemented exception because this is so common).
I would heavily argue that if you encounter such a scenario you are probably Doing It Wrong® or you were unfortunate enough to inherit/have to work with some badly designed code.
It's simply because interfaces in a framework are occasionally quite big and unwieldy, I'm sure their developer thought it was for good reasons, the other option is to split the interface up and have a class inherit from a ton of interfaces, which is just as annoying.
There's a few other replies here I'm rolling my eyes at because real-world programming is messy and being too dogmatic with ideals simply isn't practical.
(note: this is not unique to static type systems)
Edit: This is exactly the point where a type system betrayes its promises.
That’s where the “I” in SOLID comes in.
These are not "problems with static typing", though. Dynamically-typed languages have you do these things all the time, it's just that you don't even notice it anymore because it's so common and you don't have static types as an alternative there.
I have to admit for short programs without a lot of business logic where you are just providing a little glue - ie every time a csv file hits S3 run this lambda that executes a sql script to import it into a Redshift (OLAP) database, transform it with some queries, export the results back into S3, import it into Aurora (Mysql) and then load it into this appropriate tables - Python is my go to language out of the three “approved” languages (c# and JS being the other two).
I still don’t know why I like Python but hated Perl and PHP. But, I do know why and when I reach for Python over C#.
- when I know that more than one developer won’t be working on the same code and you don’t need the benefit of strong typing to help ensure that changing the signature one place won’t effect code somewhere else.
- Little conditional business logic. When you can easily gain complete code coverage just by running the script.
- small program that I can easily reason about.
- in the case of AWS and lambda, you can edit and test code right in the browser for both Python and JS. If you don’t need anything but the AWS SDK (Boto3), there is nothing to package. Once you get your code right, you copy and paste your code from the AWS consuls, export your configuration as a CloudFormation snippet and create your CI/CD pipeline.
It’s also easier to debug directly on the server without having to either have a compiler on the server or trying to set up remote debugging.
Until about a month ago, I had only written a REST API in C#. I wrote my first service in Node/Express just because I wanted to be one of the cool kids. I hated every second of not having a strongly typed language especially with my JSON objects. The tooling by definition can’t be as good especially for refactoring. Yeah I know about Typescript and I’ve played with it. I didn’t want to go down that rabbit hole and try to get buy in from the company.
On a side note. I’ve used Mongo before but only with C#. With the C# Mongo LINQ driver, you are working with strongly typed collections and strongly typed LINQ queries. I can’t imagine the hellish landscape of using a weakly typed language and a weakly typed data store.
I agree with you about there being no concrete advantage to not being forced to specify the type of something; honestly, I don't even like type inference in eg C++ all that much, except as syntactic sugar over really ugly template types for whom the type just pollutes the code without adding any signal. But for example, I don't even really see the advantage of using auto in:
for (Widget widget : widgets) {
That being said, I do believe there is a concrete flexibility advantage, that I pray you never need to take advantage of but that sometimes can't be avoided in seed-stage startups. There were a couple of times where I monkey-patched the definition of a function (including an external library once! Oh the shame...) in order to get something working for a sales demo, and we just wouldn't have had the velocity to take shortcuts like that in a language like Java or C++, though I would personally make the fixing of the hack a P0 or at least a P1. These kind of "intentional bugs" allow you a bit more flexibility with using tech debt, and in the hands of skilled enough engineers, can provide you with boosts of velocity when you need it in the short term (at the cost of overall velocity).
Similarly, some pretty nifty stuff in libraries is possible due to the flexibility of these languages. I'm not a huge fan of frameworks in general, but my understanding of Django's ORM is that it compares quite favorably to ORMs in other languages. Given the amount of times I needed to do database surgery on prod (did I mention how much this company sucked?), it was a lifesaver to 1) have such an expressive interface to the DB and 2) have both an interpreter in the same language as our code and the ability to import and run production code. I'd imagine that pretty much every part of this would be more difficult in a less permissive language.
The only ORMs I’ve ever found useful are those based on LINQ because LINQ makes any ORM (and whatever you call the Mongo LINQ driver) first class citizen and it integrates into the language.
In the specific context of stuff-is-on-fire, production database changes (I'll reiterate, this place sucked), the fluency of the ORM and its seamlessness with our code was helpful. I don't know much about LINQ, but I gather that the syntax is declarative, which is a perhaps-minor speedbump that would nevertheless have made me less fluent in the specific above-described scenario.
In the general case of engineering, LINQ looks like something I would far prefer.
This isn't a criticism of your comment or anyone else's, but I have to say I'm amused by how all of my responses downthread of my comment are in defense of me being charitable towards more permissive languages (as I said, my bias probably swings solidly the other way, and if I never engineer in a permissive language again, I'll die happy). I kind of assumed it would go the other way, with HN jumping on my comment as elitist or something.
For instance if you write the following using LINQ:
var x = from c in customers where c.Age > 65 && c.Sex == “male” select c;
var results = x.ToList():
If customer represents an in memory list, it gets translated as regular old compiled code.
If customer represents a sql database table - it gets translated to SQL and run on the database server.
If customer represents a Mongo collection. It gets translated to a Mongo query.
Of course you can do stuff like switch out what customer represents at runtime and it will be translated appropriately. You also get autocomplete help and compile time type checking.
You can pass the where clause around:
var expression = c.Age > 65;
And pass it into a method.
You pay for that "flexibility" quite directly in lack of verifiability and of adequate performance. It's not a free lunch - if it was, we would all be using LISP anyway.
What happened is that 1980s hardware was not ready for languages slower than C. C became the dominant language, and C-style syntax became the dominant syntax, with C++, Java and JavaScript following its lead. Eventually people adopted languages slower than C, but they didn't adopt paren-prefix notation.
The obvious example here is that I unquestionably do my personal scripting in Python instead of C++, despite very much enjoying engineering jobs in the latter to those in the former. Different situations, different requirements.
It's easier to write code in dynamic-typed languages because (generally speaking, I'm pretty sure there are exceptions to this rule) dynamically-typed languages are not as verbose as static-typed languages.
The focus is now on writing bug-free programs as best as we can, but I remember that when I started programming about 15 years ago there was still a vibe of "let's make it as easier as possible for people to program so that everyone can write code, bugs don't matter (at least not that much)". That was the general vibe behind HTML and browsers (the way that you can see the actual HTML "code" that displays this very page, even if it may be "broken" HTML), behind early JavaScript (anyone remembers how one was able to include different JS-widgets on one one's HTML page?) and behind very cool projects like One Laptop per Child [1] (which got killed by Intel and MS among others and which, presumably, was allowing its users to see the Python source-code of each program they were using). Those were the dying times (even though I didn't know that at the time) of "worse is better".
But in the meantime the general feeling around writing computer programs has changed, correctness and "bug-free-ness" are now seen as one of the main mantras of being a programmer, and the programming profession itself has built quite a few walls around it (TDD was the first one, "use static types or you're not a real programmer" seems to be the latest one) so that the general populace can be kept at the gates of the "programming profession". In a way that suits me, because I'm a programmer and less people practicing this profession means more potential guaranteed earnings for me, but the idealist in me that at one moment thought that programming could (and should) become as widespread as reading stuff like books and newspapers feels let down by these industry trends.
I really don't understand this argument. What typed languages have you used?
Decent type system are able to infer most or all of your types. Meaning you ge the benefits of a type system without having to explicitly type everything.
Also, when you code, is your bottleneck serious how fast you can type? If it is, I would suggest you need a different approach.
> But in the meantime the general feeling around writing computer programs has changed, correctness and "bug-free-ness" are now seen as one of the main mantras of being a programmer
We are engineers, or at least we should be. Correctness has to be a goal, although I feel like even saying that is laughable right now. The amount of effort to prove a program is correct is extremely high for anything nontrivial. Robust type systems at least help cover our bases and help us reason about the structure of things, and more powerful type systems allow you to push even more invariants to the type system.
My brain can process only so much information, the less thing it has to process (like the less type declarations) the better.
> We are engineers, or at least we should be. Correctness has to be a goal, although I feel like even saying that is laughable right now.
That’s what I was trying to explain, in a very convoluted way, that back in the day programmers were not intrinsically viewed as engineers, and that that viewpoint was seen as a valid one. Engineers were seen as building cathedrals while programmers (or hackers, if you will) were seen as building bazaars and other random stuff (like forums written in Lisp-like languages like Arc).
Ignoring that I've already mentioned inference that makes this a non-issue. My other problem with this is that just because you don't have a type system or don't have to write types, doesn't mean that the constraints of that 'information' goes away.
Your function still has a set of invariants that need to be maintained, you've just opted to make them invisible and have your program crash when they are violated. Instead of knowing statically when you've made a mistake.
I don't understand this reasoning though. Actually typing out the code takes a fraction of the time that you spend on writing software. Furthermore, we have autocomplete and had autocomplete for a long time in statically typed languages.
I was formally taught programming about a decade ago in Python, but I naturally gravitated towards Java (and C) because of static typing. The same arguments were made back then and I just didn't get them.
The problem here is that half the people on this thread are looking for a simple, low-dimensional model to cram the issue into, requiring dynamic languages to have _zero_ advantages in order to sustain their view that statically-typed languages are superior overall.
I'm not really sure how to fix this tendency, and it's by no means limited to just disputes within the field of programming, Simpler, black-and-white models are much easier for the brain to handle, so people gravitate towards them even when they don't match reality as accurately.
[1] though it is fair to say that a big chunk of the advantage of dynamically-typed languages is "accessibility to those who barely know how to code", which doesn't affect the calculus for someone who's capable of writing in either type of language
For instance, if I am writing a complex function `frobnicate(x)` and I notice that I need to pass a configuration object to the function, I can just `frobnicate(x, opts)` in the function declaration, and `x.foo` to access the relevant option, and go back to the complexity that is still fresh in my mind.
With a verbose type system, I need to pick a type for `opts` NOW, which means I need to define a new type NOW, which will typicaly involve a new file with at least a dozen lines. By the time I'm done generating those stubs, the hard bits of the original function will have faded from my mind.
The code I write, in either case, isn't the final code. Once I have an initial working version where the complexity has been embedded in actual code instead of floating around in my mind, I can start iterating, refactoring, annotating, and so on. In TypeScript, at this point I will define a new options type and annotate `opts` with it.
Typescript is not a language I would say has a verbose type system.
There are two parts to that: no required boilerplate (“Hello, world!” shouldn’t require more than twice the number of characters, and requiring a manual compile-link step is a no-no) and low time to first output (who cares that each value carries a hundred or more bits of type info and takes an indirection to access, as long a as that “Hello, world” makes it to the screen in <100ms)
Historical counterexamples to the claim people prefer dynamic typing are the ROM-based Basics of the 8-bit computer era. Statically-typed, but popular, IMO because they ticked both boxes.
> So, what kind of flexibility do you get from dynamic typing?
I'm the same, I don't get it at all. If you're going to voluntarily opt-out of automatic safety checks you better have a mind blowing list of good reasons for it.
I tend to agree, and i classify this as part of a larger attention to detail / conscientiousness / desire to get things right trait which i associate with good programmers.
> things like [...] "inability to customize interfaces" bother highly-productive engineers more than less-productive ones
This i'm skeptical about. Enthusiasm for tricked-out vimrcs, heterodox keyboard layouts, and the like seem evenly distributed across the talent spectrum. If anything, there's a weak negative correlation.
But, as with wutbrodo's, it's likely these theories are partly self-serving.
Thanks for your perspective. We may be running in different crowds but tricked-out vimrcs (or even using vim/emacs) haven't ever been that popular around me. Even at Google, there were MacBooks everywhere. There just didn't seem to be any interest in using vim or Linux to signal that you're hardcore, as opposed to actually getting value out of it. At the company I'm currently at, the person generally understood to be the most talented guy on the team I'm joining just happens to be the only one on emacs instead of VScode. To reiterate, all of this is pretty low-confidence for the reasons in my orig comment, but within my small dataset, the correlation has been quite strong.
In my personal experience: I spent an hour writing the base of a customized vimrc one morning at Google at the start of my career. In the years since then, the process has been 1) notice annoyance or repeatable process, 2) spend 5 minutes scripting it, 3) go on with work and life and see a positive ROI usually within weeks (ignoring the much more important benefit of not interrupting my thinking with manual tasks). I do something similar but much more ambitious for my desktop (currently an i3/Debian system that's almost entirely operated by keyboard). This incremental approach has worked out _really_ well for me, and the accumulated effect over the years is a system that's very, very customized to my use-cases and needs. It's to the point that, at the job I just started at, I'm a little in shock to see people stretching a single chrome window across one massive monitor and an IDE across the other, with a dozen floating windows behind each. By contrast, I have 5-7 workspaces in my working set at any time, each one of which has multiple windows in use. Again, I'm pretty sensitive to the risk of feeling productive instead of being productive by building tools that aren't worth it, but the incremental approach generally constrains me to the stuff that I'm sure will be helpful.
(There are of course, places where the value/effort graph has a big discontinuity, and I take the time probably once every ~six months to handle stuff like that, in the form of a mini-project that I do after thinking about whether the ROI would be worth it. I also often get the chance to try my hand at new things in the course of doing so, which helps shift the balance. Eg, right now I'm working on breaking the frustrating interface barrier that Chrome presents: as much as I like the web, shifting so much into the browser and away from the OS/WM has been a giant leap backwards for usability, particularly given how obsessed website designers are with mouse-heavy interfaces)
I've seen people go down a rabbit hole of optimizing tooling to the point where it crowds out using those tools to actually accomplish anything. At some point worse is better.
(I like ClojureScript and Elm more than TypeScript for web development, and prefer Python for non-web scripting, YMMV)
I work in both Python and JavaScript, and I often miss Typescript’s power. I think Python still has big advantage in the libraries.
The standard library is full featured and well documented. But perhaps just as important is that libraries are also really often high quality with clear documentation.
I think there’s a lot of interesting JS libraries and experimentation. But I love not feeling a need for better Python libs, because they’re already so mature.
I think that a “tidy verse”-style project in JS to offer a “proper standard library” across the ecosystem would do wonders for the bit of a mess server-side libraries are at the moment. Something that could be as effective as jQuery was at corralling DOM API . The Node.js APIs are good baselines but not super fun to use directly...
Python is much more traditional in that it has a rich standard library to begin with, so most third party libraries are also bigger, and it's easier to track of what does what.
I'm curious why people dislike this. I'd rather have lots of little decoupled libraries than huge monolithic dependencies that attempt to do everything in one big folder. The big libraries often duplicate functionality found in smaller tools, and I would rather they simply consume a bunch of small tools that can be shared across the ecosystem.
It reminds me of the oldschool pre-systemd UNIX philosophy of "do one thing, and do it well". Did everybody change their minds on this?
* UNIX tools don't get updated every week.
* UNIX tools don't get deprecated and replaced every month.
* When UNIX tools do update, the updates don't carry a risk of adding shady code from new maintainers because the original author "got bored lol" and handed the repo over to the first person who asked.
* When UNIX tools do update, the updates don't introduce unvetted and undocumented code that will, for example, start displaying Christmas decorations in your UI on certain dates.
How many UNIX tools actually come in "writing this from scratch is faster than searching for a tool that already does this, so I'll just write my own" size (except for `yes` and such)? Trick question: is trying to install a new UNIX tool similar to this [0]?
The jokes in the Linux "man" command that indeed did trigger at a certain magic time.
* https://unix.stackexchange.com/questions/405783/
The failure to take on board the improvements to Linux "ifconfig" from 2009.
* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=359676#41
Changes to GNU "ls" that did not go unnoticed.
One of the aspects of, ahem, bazaar-style development that we don't talk about much is that decisions get made by whoever can be bothered to turn up. If it so happens that the only people maintaining ls are the three people crazy enough to care about maintaining ls, then ls gets crazy decisions.
But there are other misc issues. For example, each package is a separate download with a bunch of metadata and overhead, so on download it spends a lot more time on those two lines than it would if they were just part of the library.
I like code splitting and there's a lot of benefits to the approach JS is taking, but it's not without its faults.
* https://news.ycombinator.com/item?id=19024256
* https://news.ycombinator.com/item?id=13390245
The actual ideas in computing when it comes to modular systems are cohesion and coupling. Modular systems should have high cohesion and low coupling.
The objection to leftpad levels of granularity is that it is high coupling, lots of little modules inevitably requiring one another in large numbers and complex ways. And a lack of any grouping at all is actually a lack of any sort of cohesion, coincidental or otherwise.
I'd be a lot happier with npm if tiny libraries took up less space on disk. Storing a local copy of each dependency's readme file, license file, package.json and test suite is overkill when the module itself is only a few lines long. Its also very common for sloppy package maintainers to accidentally ship binary files in their npm module (image banners for their readme.md, test data, etc.)
Npm should only download the javascript file for small single-file modules. Nodejs itself already supports this - if you have a javascript file in your node_modules directory, require() will pull it in like normal. The license might need to be prepended to the file in a comment block, but this could easily be done automatically.
When people need to do a thing that has already been done a 1000 times there are at least 3 ways:
(1) write it yourself
(2) copy/paste someone else's code
(3) import/vendor someone else's code
(1) is definitely not a sustainable choice, especially for something that is frequently done.
The real interesting bit here is that I think (2) is just a shitty version of (3). You inherit the benefits and the bugs of the code you copy/pasted, and likely understand that code less than (1), which is increasingly true the more nuanced (and probably performant/good) the code is.
I think the question is whether you can see your spaghetti dependency graph or not, and how big the strands of spaghetti are. It's reasonable but likely pointless to argue about the size of spaghetti.
JS's library ecosystem lays bare the actual nature of bazaar-style development. The thing about cathedrals is that they take a while to build, and even then without the right people you can build bad ones -- JS's alignment to the bazaar style of doing things is key to it's iteration speed, and is it's best tether to the UNIX model.
All that said, not having packages be immutable from the get-go, and willy-nilly updating of dependencies without manual review are indeed problems with the ecosystem, with immutable packaging being the only one anyone can actually fix with code.
https://twistedmatrix.com/trac/
https://github.com/eventmachine/eventmachine
(I guess Perl does too, but I haven't used Perl for 10 years)
I admit the promise pattern may be simpler to understand, but if you need to build a heavily evented system Node doesn't provide anything unique that can't be achieved in other ecosystems.
Also, the async IO support via libraries are there for ruby and python, but that brought along with it significant fragmentation (at the very least in the python case, I know less about ruby) -- no one was 100% sure whether to use tornado, twisted, gevent/eventlet for a very long time.
A quote from another comment I wrote[1]:
>> If we really want to get in the weeds, we could talk about python 2.x's floundering around async IO -- IIRC Twisted/Tornado/Gevent/Eventlet were all close to equally good options (and thus mindshare was split) while node had it built in, with a virtualenv-by-default packaging system. Also there's WSGI -- the interoperability it provides is cool, but also means more moving parts than a simple node+express setup.
[EDIT] - It's a little unfair of me to decry the fact that multiple good solutions existed as a bad thing when I'd say it was a strength of the ecosystem. I do think it's valid to discuss the issues with packages that weren't compatible across interface boundaries between these async io libraries though.
> (I guess Perl does too, but I haven't used Perl for 10 years)
Same here, it's been a similar duration since I've used Perl for anything but even back then, the major difference between perl and the others was extremely good native regex support and the ability to spawn system threads that actually ran in parallel without coordinating another process.
> I admit the promise pattern may be simpler to understand, but if you need to build a heavily evented system Node doesn't provide anything unique that can't be achieved in other ecosystems.
My point was about the ease of achieving it... In the last ~3 years I've seen a wave of articles in the python blogosphere about how to properly use asyncio and all these new async-enabled libraries pop up (the old ones that relied on twisted/eventlet/whatever else are still around of course and perfectly good) -- what I'm saying is that JS has had this since the get go, and it's been thrashing python/ruby on benchmarks that favor the usecase for the same amount of time, especially if someone doesn't use one of the appropriate libraries.
NodeJS's async io story is "batteries included", and that's huge in this day and age. Whether node has the right interface or if they made it ergonomic enough is a whole 'nother thing though...
Are Javascript workers something different?
That makes me think they run in the same address space, but use separate runtimes. So it's more scalable than ruby and python threads or multiprocessing.
They're an improvement over a single GIL but not by much. IIUC, they're like Racket's Places: a complete duplication of the VM but sharing an address space. The basic implementation doesn't really save much memory but it's a bit more convenient.
There's been a couple of attempts to implement subinterpreters for CRuby too.
https://medium.com/lazy-engineering/node-worker-threads-b57a...
I think most people who’ve spent a significant amount of time with strong-types do them by heart.
We didn’t leave strong-typing to leave strong-typing btw, we just started doing more and more with Python.
I did Python for embedded systems for a year. Python is so expressive and well equipped, but the lack of pre-execution type analysis leaves you vulnerable to severe defects unless your testing coverage is 100%. I would prefer Java over Python just for the safety the compilation step provides(and I don't care for Java). I love programming in Python, but system reliability trumps my personal desires in this case.
- https://docs.python.org/3/library/typing.html
Also flake8 has also helped -- it's just a linter but using it in conjunction with the typing module and running that in a watch loop in another terminal window has done wonders for me.
I love flake8. Properly tuned, it can be a great help. It saved my butt more than once.
Both Java & Python are strongly typed..
e.g. in python, <3 + "2"> gives <TypeError: unsupported operand type(s) for +: 'int' and 'str'>
How do you know this? It's exceedingly rare that anyone actually keeps track and monitors rate of defects in any application, but there have been studies done[0][1] on the ability of stronger typing in languages to reduce errors.
My condolences for your time using Java and C# -- they are often what people first consider when they think of languages with "good" type systems, but I dislike them both because what they introduce as "good" type systems are often very much not, in comparison to languages like Haskell.
It's a bit disingenuous to act like Haskell has been a performant/production-ready option as long as those languages but still, almost every time I talk to someone and we get to the topic of typing I almost always assume that their bad experiences came from those languages.
> We didn’t leave strong-typing to leave strong-typing btw, we just started doing more and more with Python.
What are your opinions on https://docs.python.org/3/library/typing.html ?
[0]: http://ttendency.cs.ucl.ac.uk/projects/type_study/documents/...
I think we’ll see more and more use of the typing library, especially on big projects. Similarly we’re having debates on typescript for front-ends, not because we need it right now, but because it’ll probably really suck if it turns out we were wrong about not needing it.
What you're saying here and in your previous comment makes sense. From my experience, engineers who have a LOT of diverse experiences with different paradigms tend to agree that types don't reduce bug density in the code.
Types are a learning tool like training wheels on a bike; with comparable costs in terms of efficiency. Don't get me wrong, training wheels are great but you won't find them at the Olympics.
That is such an idiotic comparison, I don't really know what to say here.
1.) Proper type inference means you don't have to explicitly write out most/any types, so there is no "cost in terms of efficiency". Some annotations might be required from time to time. If the type checker throws an error that means that your types don't align and that you further reason about your program - which is something you need to do in dynamic languages as well, just without any assurance in the first place.
2.) Types allow performance optimisations not available in dynamically typed languages, so again - no "cost in efficiency".
3.) In dynamic languages you don't get as much help from your tools as in static languages, so you can't exclude whole classes of bugs that your compiler finds for you.
So, where does this "cost in efficiency" occur? How is static code analysis a "training wheel"?
But it is.
Wow! No!
If this is what you feel, you might have been doing unit testing wrong the whole time! Unit testing is a powerful technique that is useful regardless of your language. the first unit testing libraries and frameworks started with Statically Typed languages.
For example, if you write a function that only makes sense on numbers greater than zero, then make it take a `GreaterThanZero<Int>`. It's a contrived example but people often pride themselves on writing unit tests that check for "out of bounds" cases like that, and defensive coding in dynamic languages usually starts with "make sure they gave you the right things and nothing is null".
There are cracks to slip through with Typescript (it's not as good as using a language that enforces this stuff down to the runtime -- i.e. haskell), but in general you're going to avoid these kinds of errors in the first place if you write sufficiently useful types.
Another hot take:
The value of different types of testing is as follows (most valuable first):
- E2E tests that test customer flows
- E2E tests that prevent regressions (made after a bug was discovered)
- Smoke tests
- Integration tests
- Unit tests
Developers who believe specifying types doesn't have any drawbacks are still early in their careers :D
But with the rise of JIRA and the pressures to close tickets with the least amount of work a lot of newer developers don't realise this or are not allowed to spend the time to do it right the first time.
Python always had "types" of course. What Python 3 added was type hints (an approximation of static typing, if you will).
null/undefined is an inhabitant of every type in typescript, I would suggest purescript instead.
> Python relatively recently added typing
I think you mean the ability to add explicit types to variable names. It always had types.
> "extensive" unit testing is a poor man's type checker
Certainly.. iff you use dependent types.
> developers that don't think specifying types is helpful
Why specify types if they can be inferred? Why do work that our computers can do for us?
If you want absolute certainty in your types with TS, you should be using the `--strict` flag, which makes it so this is not the case.
For example (included with --strict):
https://www.typescriptlang.org/play/#src=class%20Base%20%7B%...
(I like TS, I just don't think it fully solves this problem!)
You're right -- but I've found in practice that strict null checking[0] is more than enough. Also I assume that Purescript has a similar bottom value as haskell[1] (please correct me if I'm wrong).
> I think you mean the ability to add explicit types to variable names. It always had types.
Yep -- I actually meant literally the `typing` module[2].
> Certainly.. iff you use dependent types.
You can get some good guarantees with container types (`newtype` in haskell terminology) that are constructed in a very specific way, assuming you have immutable values. For example, defining some type `NonZero<Int>` in typescript w/ `private readonly` interior value.
> Why specify types if they can be inferred? Why do work that our computers can do for us?
You're right -- I misspoke (misstyped?). By all means if inference is possible that is also fine.
Somewhat seperately, the statement stands as far as type signatures (i.e. `doThing :: Input -> Output` or `function doThing(in: Input): Output`) go -- they're great benefits to a codebase in my opinion.
[0]: https://basarat.gitbooks.io/typescript/docs/options/strictNu...
This isn't true as far the way you stated it. TypeScript types are not nullable by default.
How can you use existing JS libraries without supporting null anyway?
This fits python, perl, sh, node
It does not fit C/C++/C#/F#/Java?/Go/Swift/Typescript. Those are 2 steps at least. One to compile which generates an executable. Another to run the executable.
But I'll acknowledge I made up that definition. No idea of others agree. I get that it's arbitrary. I could certainly make something that given "#!/bin/execute-c-as-script" takes the .c file, compiles it to some temporary place and runs it. But that's not "most common usage" so I'll stick to my point as part of the definition of scripting languages. For me, if it's most common to have a manual compile step separate from a run step then it's not a scripting language.
Go compiles fast enough that you can use it with a wrapper "interpreter": https://github.com/erning/gorun So does C: https://github.com/ryanmjacobs/c So does Haskell: https://stackoverflow.com/a/8676377 So does f#, via `#!/usr/bin/env fsharpi --exec` I think
You can compile python: https://github.com/dropbox/pyston You can compile JS: http://enclosejs.com/ I wouldn't be surprised if someone found a way to AOT-compile bash, but I'm not going to bother looking for that.
The only criterion I've found which makes some sort of sense is whether the language designers consider "scripting" a use case to actively support and use to drive language features, or if it's a second-class citizen.
Weirdly enough, JS is the best in my opinion precisely because it does so much to not be itself. The crazy iteration speed of the JS ecosystem means that people have done crazy things, but the good stuff has remained. Python relatively recently added typing[0], and Matz has mentioned that ruby has no plans to include types in the language proper (though people are working on it). Perl might be the closest in terms of features since it has thread support but...
If you enjoyed this hot take, here are a few others you might enjoy:
- "extensive" unit testing is a poor man's type checker
- developers that don't think specifying types is helpful are still early in their careers (those who suffered in the dungeons of Java get some slack)
[0]: https://docs.python.org/3/library/typing.html