The author’s biggest criticism is that Julia doesn’t have tooling to make the developer experience better. There’s Revise, JuliFormatter, LanguageServer and Jet, but the development experience in Python is enviable. There’s like 3 different REPLs, at least two competing linters and auto formatters. It’s okay to admit that these are places Julia is lacking.
I think your kind of response to criticism about Julia is what gives the Julia community a bad name, in my opinion. What is wrong with saying these things suck and need improvements? Would you rather Julia not improve and stay the way it is right now forever? Surely I hope not.
The subjectivity arises in deciding whether some third party package Y is totally common, essential, de facto part of the language itself, or esoteric, negligible, who cares. I perceive that a lot of the back and forth around "is Julia good" basically boils down to this.
"I don't like Julia! I tried to use it and <package> had an annoying bug!" "Okay, but <package> isn't part of Julia itself or even written/maintained by core Julia contributors, so bugs therein are irrelevant for the awesome multiple dispatch speedy beauty of Julia!" "Hard disagree, <package> is the de facto standard for <common task Julia should be good at> and maintained by <important prominent figure in the Julia community. If even such a high visibility package is broken, the whole ecosystem must be rotten!"
The parties are just talking past each other here; there's no ground truth. We'd all be better off if we were more explicit about whether comments applied to (a) the core language itself (b) core language and standard lib (c) the universal experience of working in Julia, i.e. packages that nearly everyone encounters (or lack thereof) (d) packages which are key for certain kind of work but irrelevant for others; auto differentiation would seem to go here (e) niche packages.
Disclaimers?: I love Julia-the-language; it is my favorite programming language by a mile. My brain seems to work similarly to Julia-the-language, so programming in Julia feels fluid and effortless to me; no other language sparks joy the same way. But Julia-the-ecosystem certainly has its holes and weak spots. And the limited size of Julia-the-community means even relatively prominent packages can be less maintained than e.g. Python analogues. Example 1: it's unnerving the extent to which the Julia data ecosystem rests on the heroic efforts of a few people [1]. Example 2: even relatively mainstream packages can have issues and PRs unaddressed indefinitely, e.g. I like to plot with Gadfly.jl but am nonetheless frustrated that I've had an open issue there and an upstream PR for over a year.
But on the other hand, all the reasons the author has listed as annoyances I've experienced the exact same things and more. And I'm definitely not blaming the package maintainers for this. There's either 1) something wrong with the Julia language that results in code that is harder to maintain at scale, or 2) missing tooling to help package maintainers solve some of these problems.
In my personal opinion (and also almost universally the opinion of 20-25 of my colleagues some of whom are way smarter and more experienced than me), it is a little bit of both. Tooling like JET for linting is just taking off, and for years people have been writing and maintaining packages without linters. The ONLY way to ensure the Julia code you write is to write LOTS and LOTS of tests. In Python, if I use type hints and run mypy a whole class of errors is avoided. That's not the case in Julia. At work, we maintain a close to 10,000 line Julia code base and honestly if we'd known the tooling was going to be the way it is, we'd not have chosen Julia.
The fact that startup time is so painful + coupled with the fact that tests are basically the only way to guarantee that you are writing correct code, makes it EXTREMELY painful to work in a large team. Our internal CI takes hours to run for even the smallest changes. Even a small scale refactoring takes weeks of review from everyone involved and it is a time consuming frustrating and expensive process for the organization.
I've worked with similar size codebases in Python and C++ and have had no where near this kind of difficulty.
I can't find the link to the talk right now, but Bryan Cantrill talks about "values" of a programming language / ecosystem. And based on the way Julia is designed and implemented, I'd say the "values" are "run time performance", "scientific applications", "research suitability" (ease of creating environments, DataFrames.jl for data science etc). I think "correctness" as a value is FAR from the priority in this language, which incentivizes researchers to write code that gets the job done quick but fails to be long term maintainable or viable.
The worst part is I don't know how Julia will ever solve these issues. Leveraging LLVM to generate fast machine code at runtime is the very basis of the language, so you'll almost always (unless static compilation happens) pay a penalty at start up and first run. Using the type system for dispatch means that static analysis is just going to suck. How JET works is a mystery to me, so I'm not sure what the extent of the improvements there can be, but based on talking with others in the Julia community, I don't think it'll ever rival Python's or CPP's static linting capabilities.
The Julia devs are constantly working on improvements, so I expect things will get marginally better and maybe in 5 years or so it'll be a tolerable experience. But all in all, I'm not holding my breath.
Based on the above, I expect more than marginal improvements in these areas. Would you agree?
I heartily disagree with that. Bikeshedding is about focussing on the trivial, and the symbols we choose in our codebases are hardly trivial, and indeed many of us regard naming things as one of the central problems in programming[1].
[1]:https://medium.com/hackernoon/naming-the-things-in-programmi...
It is bikeshedding because it is analogous to insisting that using "angle" instead of "θ", or "radius" instead of "r" in a 2D geometry library is superior and takes your code from being a lackluster to something that shines (in the words of the original author), while not having anything useful to say anything about the mathematical/technical aspects of the code itself.
Here is the definition of bikeshedding:
> The term was coined as a metaphor to illuminate Parkinson’s Law of Triviality. Parkinson observed that a committee whose job is to approve plans for a nuclear power plant may spend the majority of its time on relatively unimportant but easy-to-grasp issues, such as what materials to use for the staff bikeshed, while neglecting the design of the power plant itself, which is far more important but also far more difficult to criticize constructively. It was popularized in the Berkeley Software Distribution community by Poul-Henning Kamp[1] and has spread from there to the software industry at large.
Sure, there'll be a subset of users of these libraries that have read ML/textbooks and are familiar with what η means in this context.
Today, many (most?) users of ML libraries will probably not know what η means without looking it up. Adhering to mathematical notation puts up an unnecessary barrier to using the API/code and ultimately limits wider engagement/collaboration.
To attract a bigger slice of the ML community, choosing names that the ML hobbyyist can read, understand and use without pause is the better path forward.
FYI, the documentation of the function https://fluxml.ai/Flux.jl/stable/training/optimisers/ explicitly says it is learning rate:
> Learning rate (η): Amount by which gradients are discounted before updating the weights.
so this is already explicit to anyone who reads the documentation. The quibble in the post is about the named parameter.
You can look up "learning rate" much easier than to look up "what is this Greek letter on my screen" followed by "what is the use of this Greek letter in my context" and only then followed by searching for "learning rate"
More importantly, it's possible to know what a learning rate is without knowing what Greek letter it's commonly denoted as. Especially since mathematical notation is so inconsistent across authors. I want less ambiguity in code, not more. Explicit is better than implicit.
Mathematical notation is notorious for being an absolute mess of inconsistencies. Who in their right mind looked at it and went "yep, I want more of this in my source code".
For research-focused code, it is likely that whatever you're implementing was initially described in terms of mathematical notation (e.g., in a paper or book). It can be helpful to have variables that unambiguously match that canonical source. In fact, a lot of my Julia code has docstrings containing references/links to the original paper and a comment noting that it uses the notation therein.
This sidesteps the problem where textual descriptions like `learning_rate` can sometimes be ambiguous: is it the original learning rate, or perhaps the current rate after applying some sort of schedule or decay? I think the Flux documentation is pretty close to ideal, in that it's got a symbol you can match against equations (though no reference to them) as well as text that you can search to learn more.
As I said, the necessary keywords for Googling it, along with a brief description is already present in the documentation.
The quibble here is about the necessity of reproducing the all the necessary keywords for an accurate Google search during every single function call.
> Mathematical notation is notorious for being an absolute mess of inconsistencies.
According to whom? What exactly is inconsistent?
No, that is not the quibble. My quibble is with choosing identifiers that make code less legible when taken by itself. The best code teaches future readers about how it works.
The greek letter is used in the documentation. And the reason is that every optimiser's documentation links to the original paper, and tries to follow that. If the Adam paper calls the two decay rates β1, β2, then staying close to that seems the least confusing option.
My most general point is that the identifiers we use in our code are almost never just convention or taste when we are sharing that code with anyone else (and for most, "anyone else" includes our future selves). Getting a little more specific, I'm specifically interested in Julia and look forward to working in it further, but I've personally felt pain around scientific/mathematical notation when trying to understand code I've found on github. tagrun dismissing my pain as nonsense and the people who argue for my ilk as perpetrators of bikesheddding is dipshitted. Yeah, I'm probably the asshole for being a college dropout trying to leverage modern scientific computing for my own ends (snoogins), but I'm also willing to bet tagrun is probably the member of a team that talks down to junior members and complains they haven't read enough papers or the right papers to see the magnificience of their code ;).
It's fine to write code that demands a domain expert to understand, but don't pretend like its good across all dimensions. There are tradeoffs involved.
Personally I find the preponderance of scientific/mathematical notation (whatever you want to call it) in Julia to be cute; It certainly does bind the code to linked papers in a pretty cool way when it all fits together properly. That said, its a pain in the ass when it doesn't fit together properly and I've personally had a journey into Julia spoiled due to frequency at which I had to figure out how to notate something or what word to use when regarding some squiggle I haven't encountered before. I look forward to having a better intuition for the greek alphabet but until then Julia will often be harder to read, let alone understand when compared to ruby or javascript or go or C# or any other of the roughly dozen programming languages I've worked with and feel comfortable translating between.
> so this is already explicit to anyone who reads the documentation. The quibble in the post is about the named parameter.
As far as I can tell it's a documentation complaint. He has to remember "η" from the line with the signature, past the line "Gradient descent optimizer with learning rate η ...", and a heading "Parameters" until the line quoted which explains this in full.
He says this is the API, but that's inaccurate. The API being explained is that the first positional argument is the learning rate. It's not a keyword argument, so you cannot supply it by name. What variable names are used in the code is private, and in fact the struct's field name is `eta` so that you can access it without typing greek.
If this makes the top 10 list (even the top 10 list of documentation complaints) then Flux is doing OK. Especially the top 10 list of a guy with a PhD in a mathematical field. (From the sort of university which used to require students to know latin & greek, too.)
Why should reading a paper on stochastic gradient optimization be a prerequisite to understanding your idiosyncratic choices for identifiers? The fact of the matter is that I can understand code much better than acedemic prose. I'll learn the code and then supplement with the paper as needed. By using idiosyncratic identifiers you're gating off your code from people who haven't jumped through the same specific hoops you have and have the same mental muscles you have developed.
> It is bikeshedding because it is analogous to insisting that using "angle" instead of "θ", or "radius" instead of "r" in a 2D geometry library is superior and takes your code from being a lackluster to something that shines (in the words of the original author), while not having anything useful to say anything about the mathematical/technical aspects of the code itself.
No. To someone who doesn't have an established mental muscle for mathematical notation it is analgous to using thai script to write for an audience that primarily reads english: I can still use google translate, but the cognitive load is much higher to members of the audience who are native thai. That isn't bikeshedding, that's caring about understandability.
> You are saying most people don't know what η in that context means (=people who likely haven't read a book or a paper on stochastic gradient, and don't know how it actually works), but they would somehow magically figure out what it actually does if we call it "learning_rate" in ASCII letters. How does that work?
I learn from code a whole lot faster than I do from acedemic prose. I usually start with code then read the whitepapers the code refers to as I go. I learn slower from code that uses symbology I'm not familiar with. In the context of learning a new code base, unfamiliar symbols are bad in several ways.
> so this is already explicit to anyone who reads the documentation. The quibble in the post is about the named parameter.
My quibble is with the assumptions I think you make about what comprises good code quality while at the same time having suffered through code from people who share your attitudes. Naming matters to me in more ways than you're apparently versed with. The article I linked is just one small discussion on naming but not comprehensive by any means. I was linking it more in hopes that you would do further thinking of your own about a pretty wide subject. Just my opinion.
Furthermore, the page on FluxML demonstrates the problem I'm referring to. Just down from where you linked you'll find an entry like `RMSProp(η = 0.001, ρ = 0.9, ϵ = 1.0e-8)` in which `ϵ` is described nowhere in the entry. It's a random symbol that I understand to be used usually in set membership notation, but in this context (the context of some random link some random person posted in the interwebs) I have no clue what it means and thus it is a barrier to my understanding.
This was added recently, and for some reason the PR (1840) didn't fix the docs, which is bad. The Optimisers.jl version has an explanation: https://fluxml.ai/Optimisers.jl/dev/api/#Optimisers.RMSProp
> Item #2, again another Zygote bug.
If flux chose a buggy package as a dependency, that's on them, and users are well justified in steering clear of Flux if it's authors are not in habit of auditing the depencies they pull in. As of today, the Project.toml for both Flux and DiffEqFlux still lists Zygote as a dependency. Neither list Enzyme.
https://github.com/FluxML/Flux.jl/blob/master/Project.toml
https://github.com/SciML/DiffEqFlux.jl/blob/master/Project.t...
For DiffEqFlux, it's just for backwards compatibility to Flux. DiffEqFlux is a weird library because it's been "eradicated" over time. There was a point in time where it defined some necessary things to do the things in its documentation. At this point, for most tutorials it's not even required that you have the DiffEqFlux library to do it. That makes it a rather odd library haha. The transition is:
- GalacticOptim.jl has become a fully-fledged optimization package formalizing some of the heuristics and multi-package support it was using internally. https://galacticoptim.sciml.ai/dev/
- The adjoint overloads are handled all by DiffEqSensitivity.jl at this point. If you try to use them without having that library then you get an error asking you to install it. DiffEqFlux.jl just reexports it. This is why you don't see the Enzyme.jl dependency.
- Enzyme.jl has gotten better and better over time, and has become the go-to for DiffEq. In fact, we use a polyalgorithm under the hood that tends to prefer Enzyme.jl and ReverseDiff.jl for the autodiff over Zygote.jl, so it's weird that in 2022 any comparisons still feature Zygote.jl given that, internally, it's almost certainly not using Zygote. Zygote is what the user sees and interacts with, but it's not the core. (Even then, that will be changing soon with the coming Enzyme.jl overloads)
- FastChain was a setup to get around issues with Flux, but that has become a full-fledged library of its own, Lux.jl, which isn't completely done yet but will be how those pieces get deleted.
So yes, the state of DiffEqFlux.jl is that it has been a fast moving library to the point of its own destruction, basically just holding what would become "hacks" to make the high end work that were then slowly absorbed to become things that "just work" when using Julia. What the library is pivoting towards now is just being a high-level interface for common machine learning use cases of differential equations, like for defining FFJORD or continuous normalizing flows architectures, which you could do from scratch but it's nice to have somewhere that these are all defined. What to do with those tutorials, who knows, move those to DiffEqSensitivity.jl docs, and then we need some kind of inter-module documentation so that way people can easily see the 25+ docs of SciML in one website (or whatever the number is).
But honestly, while it becomes a documentation mess to eradicate a higher-level library like this, this has been our dream with Julia over the last few years. Having no place to describe the differentiability of solvers means differentiable programming is truly working. With Enzyme's improvements and such, we're supporting even things like mutation, which is progressively making almost any Julia code you write "just work" with automatic differentiation. No hacks are required to make it work with some underdocumented sublanguage (cough Jax). As everything becomes automated, it becomes harder to document it as a feature because it's instead just a property of being a Julia library.
Item #2, again another Zygote bug.
Item #3, which package? That sounds like an hyperbole, an extrapolation from a small sample, and "I'll avoid naming names" is a lazy excuse that would hide this. It is similarly easy to point to poorly written (or poorly documented) JAX code or Python code as well, so that doesn't prove that "Julia is lacklustre and JAX shines". Also, as an academician, I strongly disagree that learning_rate=... is better than η=..., but that's a matter of convention & taste, with no bearing on the correctness or performance of the package/language. It's bikeshedding. I agree that errors are usually not "instructable" in Julia ML packages (which needs to be improved), so monkey typing is less likely to succeed.
Item #4 is such a nitpicker. Sure, Julia may not have a special syntax for that particular fringe array slicing like Numpy does and you'll instead need to make a function call, but matrix/tensor code in Numpy is usually filled with calls to zips and cats with explicit indices, whereas in Julia, no explicit indexing needs to be done usually. Also, one can nitpick similarly in the opposite direction: Julia has many language features lacking from JAX or Python, why not talk about those as well?
I'm not a huge fan of Julia either (mainly because of it's garbage collected nature), but this is such a low-effort criticism of it.