> with high confidence that it will work and last a long time

And we need more of our software to be like that. Now if we could just do something about the constant pressure to compromise.

Certifications and more flexible laws for returning it when it doesn't fit the purpose, will do it.

>>But if you're under pressure to build lots of small, partially documented, constantly changing parts every day, you might be better off using more flexible equipment and materials (balsa wood, bailing wire, masking tape, etc.) to get the job done, even if the result will be messier and more prone to breaking.

Python & Deno/Node fit the bill for the latter category. Would you consider Julia in the former or latter category for ML use-cases?

For ML/scientific computing, it can fulfill both roles, I'd say. Julia looks about as flexible and easy to use as Python, R, and matlab for experimenting and iterating quickly with throwaway code, and superior to those languages in every other important dimension except one: it lacks Python's gigantic ecosystem.

Julia has a much larger ecosystem in terms of things like numerical linear algebra, scientific computing, and differentiable programming. How do you use block banded Jacobians inside of ODE solvers? Python's ML ecosystems just barely got non-stiff methods, so advanced accelerations are fairly far away, whereas these are things that have worked with Julia's ML solvers for a long time now.

Before responding, first let me say thank you for all the work you have done and continue to do. Also as I wrote above, I view Julia as superior to Python in every other way. :-)

By "ecosystem" I mean much more. PyPi currently offers around a quarter million Python packages, versus thousands for Julia. Github code and activity using Python is also currently around three orders of magnitude greater than for Julia. Python is currently the fourth most popular language on Stack Overflow (after JS, HTML, and SQL). It's a gigantic ecosystem that has evolved over three decades.

Yes true. I think it's a lot like the old Android vs iOS stuff though: counting the number of packages doesn't mean all that much because so many have been abandoned that Python probably has a much higher percentage of junk out there. That said, Python's ecosystem is still larger, but that doesn't mean it covers all domains well. I can name a bunch of holes that it has off of the top of my head, and same with Julia too, so it's not like either supersets each other. So I just think the "but Python has a bunch of packages" is a lot more nuanced than that: in many areas of ML or webdev it is much more developed, but when you get out of that realm it can get sparse in some ways.

It's not just packages and repos, but also installed base, developers, forums, discussions, corporate backers, etc.

Otherwise, I agree that not all domains are well covered by any language :-)

Something that I think is worth discussing is that julia has far more package developers per user than Python. That may seem like a weird thing to prop up, it seems to just suggest that Julia users more often find themselves needing to implement something themselves rather than use a pre-existing library, and that's definitely true.

However, Julia also makes the experience of growing from a package user to a package developer basically seamless. A big part of what makes this seamless is that almost all julia packages are written in pure julia and thus relatively easy to read and understand for a julia programmer. This coupled with the strong introspection capabilities makes it so that users who would never be writing their own packages in Python end up contributing to the Julia ecosystem.

Coupled to this is the fact that julia is a highly composable language meaning that it's very easy to combine functionality together from two separate packages that weren't designed to work together. This composability makes julia's fewer packages have far greater leverage and applicability than Python packages and also makes it so that when you do need to make your own package, you're less likely to be reinventing the wheel every time.

Consider the fact that all the big machine learning libraries in Python all have basically implemented their own entire language with their own special compiler, all the requisite datastructures, etc. In Julia, that's rarely necessary. Flux.jl is just built using native julia datastructures and julia's own (very hackable) compiler.

Great points. Agree with all of them.

Interestingly, I see the same trends with Rust -- high ratio of packages to users (already ~43K crates, despite having only 3% share of Stack Overflow last year), easy packaging (`cargo publish`), and high composability (e.g., via traits, type parameters, etc.).

An obvious question is, why does Rust have 10x more packages than Julia already, and almost a fifth as many as Python? I think it's because from day one Rust has appealed to a broader group of developers (anyone seeking better performance and/or concurrency, with memory safety and modern language features like algebraic data types, will consider Rust). Julia, in contrast, has always targeted the scientific computing community, which contains a much smaller group of developers who seem mostly content with existing tools.

I think Rust scratched a deep itch by targeting people who are unhappy with C++ and definitely started off targeting a bigger, less competitive market than Julia.

Julia has a harder marketing job than Rust. Julia needs to convince people who use $slow_dynamic_language and $fast_static_language together that they'd be better served by just using Julia for both jobs.

Even more importantly though, julia needs to convince people that it's premise is even possible. People have deep seated biases that make them unaware that it's even possible for a dynamic, garbage collected language to be fast.

Rust on the other hand 'just' has to convince people that it's a better ~1:1 replacement for C++. Most people who have written C++ deeply believe that a better language is possible and yearn for that language.

Python, R and Matlab users typically don't believe that it's possible for a language to do what julia does.

Furthermore, Julia did initially spend a lot of marketing effort on the scientific community, which is somewhat small, and is more composed of people who just see their language as a tool that only needs to be 'good enough', so they're less likely to want to switch than say systems programmers who spend all day faced with C++'s inadequacies.

I think an even bigger aspect of this is that C++ users are technical people who make technical decisions. If Rust is a better tool for them, they will switch. For a lot of scientific programmers, or even just a lot of general Python users, they don't necessarily choose Python because they know 20 languages and think Python is the best tool for the job. A lot of people use Python because... they use Python and it's what they were taught. That's a much harder audience to go to and say "would you change to this technically better language?". Most just say "I'm not really a programmer, I'm a researcher/scientist/etc. so I shouldn't spend time learning more programming", and that makes it fairly difficult.

Agree.

Speaking from personal experience, I would add that some people use Python, not only because that's what they already know, but also because... everyone else in their field is using Python and its ecosystem. New advances in their field occur largely within the confines of that ecosystem. It's hard to leave the pack behind.

Large ecosystems have significant network effects that act as barriers to new entrants.

Yeah definitely. That's what I was trying to get at in the last paragraph, and you might be right that it's an even bigger aspect than the other things I mentioned.

Additional considerations here:

Rust invested in a package manager very early, and specifically designed things to make it easy to publish packages.

Rust has a smaller standard library, and so you often need packages to do things that you may not need in other languages, which encourages the use of packages, which I think encourages the publishing of them as well, not to mention that there are gaps that would get filled by packages.