This means there is an upper bound to how good the editing and refactoring tooling will be. I suspect the best will be a little below Pythong's tooling, because Python's class helps in disambiguating methods.
One feature of Julia I think could be improved upon is the `include` statement to import code, similar to C's `include` macro directive. This makes finding functions within a package an absolute nightmare. I think this also restricts LSP's ability to find definition and usage since this tends to make Julia packages one big source file as far as the compiler is concerned. I saw that in Julia's compiler code, the core developers include C source files, not header files mind you, in other C source files, so perhaps the feature stems from their personal preference. Python's file-module may be too restricting, but Julia's including is too loose.
I feel that most of your points can be addressed if the compiler's "abstract interpreter" (static analysis) procedures were exposed and reliable/stable. Some packages (e.g. Mjolnir.jl [0]) attempt to re-implement it. Others (e.g. JET.jl [1]) try to hook into Julia's undocumented/private compiler internals (CoreCompiler). Rightfully so, only the most intrepid are willing to do this.
I've ranted about very similar things for some time now, and had some fruitful discussions, and I even offered to help to try to scratch some of the itches me and others are suffering from. I've now mostly given up too.
It's a bit sad for me. I think Julia has many things right and the core concepts are potentially revolutionary. But for me it seems that Julia ecosystem is a bit too jealous of their discovery to let it free. Weird.
https://discourse.julialang.org/t/why-no-base-iterators-map/...
The "bubble effect" here was assumption that everyone has internet access and dependencies to external ( community maintained ) packages are zero cost.
Happily 2 years later this was resolved to my satisfaction.
I'm a researcher but I have a background in software development, and especially open source. Academia is way behind software in openness, which is rather bizarre as the institutional role of academia is usually seen as providing new ideas and challenging old thinking so the community can prosper. There's some kind of a bug in the current academic culture that causes it to freeze up when it should fulfill its promise to share its findings with the world.
Probably a good way to debug this thing is to follow the money and why on earth do we have these weird hats and robes?
Julia's optimal sorts of workflows are different than many expect — especially for folks coming from static languages. Julia is a dynamic language. But also, Julia isn't Python. There are ways in which your workflows from other languages just aren't optimal in Julia. So when someone comes in "hot" and rants about how their workflow isn't working out how they expect, others jump in and — yes, defensively at times — point out alternatives that work for them.
I think some of this tension comes from an expectation mismatch. Some of the changes noted in these comments about how Julia is presented on julialang.org were made specifically due to this sort of expectation mismatch.
This doesn't mean that nobody in the community cares about improving workflows — in fact I know that's not true. But if you want to use Julia today, there are definitely some happy paths that we should guide folks towards.
And that's exactly what I've seen over and over again, in different communities, such as the ones I've mentioned, this attitude of "you're holding it wrong".
No language or tool can be everything to everybody, that's perfectly reasonable. If the goal of Julia is different from that of people that want to run applications in production—or at least those concerns aren't the most important ones—fair enough. But all too often (and I'm not necessarily talking about Julia here; I haven't spent enough time on the Julia forums, I'm mostly referring to some of the other examples I've cited) real concerns are just dismissed out of hand.
I also don't believe that enforcing certain very specific workflows at the exclusion of any others is necessarily a great way to become a language that lots of people want to use. People come from different backgrounds and have different preferences and needs (sometimes for very good reasons), successful languages like Python recognise this and let you work with a wide range of tools.
Finally, I'm just very skeptical about a certain kind of NIH syndrome that I've seen in some of these communities, something like "encapsulation? well, all these other languages might need it, but we actually don't, because XYZ" or similar things.
You're really reading this wrong. The issue is moreso about personal knowledge. I see this on the Discourse sometimes where someone comes blazing in saying "I want to use PyCharm for Julia and I run into ...", "Special Lisp-only IDE has a Julia plugin and it fails because ...", or "I have this special terminal setup and I no 0 latency and ...". The answer of course is, if you need help, use VS Code or Juno/Atom. Those have a larger userbase which that has fixed exactly those issues and has a helpful dev community on those site which will get you up and running. That's not "you're holding it wrong", that's people trying to help you into an area where they know how to help you. You can shrug it all off and blame it on them, but at the end of the day, people are sharing what they know works, a reliable solution they built for exactly this problem, and then are getting flamed on the internet for having "NIH syndrome" for not fixing some random IDE plugin they've never used. I just don't see how you can expect every person to be an expert on every possible workflow you can imagine and start throwing flame when someone tries to lead you to water.
In the Julia community in particular you tend to have people who are more experts in their scientific domains. I am in that category: I can help you in anything ML, high performance computing, differential equations, etc. but I use Windows and the Juno IDE. If you open an issue about something weird happening on emacs, I wouldn't even know how to get it installed, sorry, that's not my expertise. The Julia community has a lot more people in probabilistic programming than it has Javascript devs, and so people share how they know how to fix your problem "can you run it in VS Code and show the profile?", not "here's how to run some daemon server thing" etc. And that's perfectly fine, it has its advantages and disadvantages.
And yes, that destination does include production.
Julia is hard to debug as is (and this is fully acceptable to me due to the architecture) and these hacks on hacks make it just impossible.
Julia suffers from the same "fancy calculator" approach than e.g. R and Matlab, and like those, they refuse to take note of the decades of experience in software development that this just leads to a total mess if you have anything more complicated than a one-off analysis. It's like insisting on editing files with ed because that's how it's always been done.
I think your view of Julia as a "fancy calculator" may be part and parcel to the difficulty. I do use it as a fancy calculator, but I also use local packages all the time — and that's where you'll find the best success in both tooling and code structure for sustainable (and modern) development. We need to do a better job of helping folks more effectively develop and work with Julia before it becomes frustrating.
My view is that Julia (and R and Matlab etc) ecosystem's view is a "fancy calculator". It's hard to integrate to anything else, it almost actively tries to make e.g. unix-type workflow difficult (not much CLI, everything done in REPL, e.g. package management). Not as bad as e.g. R where its almost impossible to just run a file of code instead of doing a "workflow session".
Perhaps there is some disconnect in how different parties of the discussion see the whole thing. For me this sounds a lot like "you're holding it wrong". What background are you coming from and what languages/programming environments you are familiar with? I get the impression that you don't have much general purpose programming background and somehow tend to assume that people are using "wrong workflows" due to being just stupid or something. But this is how it comes off to me, I think you mean something else, but I don't know what that is.
My approach is: I want files with code and I want to run some of those with some kind of entry point. And preferably organize it to files or something like that. This is the usual setup for general purpose programming languages. In general purpose programming you rarely tend to stick solely to one language, any larger project is usually a mishmash of different stuff, e.g. glue code in python, tight loops in C, maybe some bash thrown around to do some more trivial chores.
This is the overall structure of practically all general purpose programming languages. This requires interfacing with these languages (often done in bash or something in unixy systems at least) and tools that work regardless of language (e.g. version control, grepping, Make etc). Julia (and R and Matlab etc) just plain refuse to even consider this as a valid approach. If you mention it somewhere, you tend to get some bizarre instant putdowns that provide no real rationale, i.e. "you're just holding it wrong".
In programming I tend to think the whole computer and its OS and so on as the programming tool. My impression is that for Julia folks Julia is something totally separate from anything else in the computer. But I don't understand why. Is it that its felt that changing syntax on the fly is too cognitively demanding or something? This is something that one gets used to very fast. And also tends to ease the IMHO weird hung-ups on syntax. In fact, I think Julia has made some (IMHO misguided) syntactic decisions because they just want to do something differently from Python just for the sake of being different. I find this totally senseless.
Us CLI folks are not even asking for much. Just a way to call a program with arguments and so that you don't have to do compile identical code every time you do something. That's it. Is that unreasonable? It wouldn't take anything away from people using REPLs and Notebooks. We don't want to change the language, the language is fine and that's why we'd like to use it but just cant because these paper cuts!
I've expanded on my rationale for my workflow and how Julia makes it really difficult e.g. here, where I admittedly started with too strong wordings out of frustration: https://news.ycombinator.com/item?id=26134970
No, you're not stupid. You've correctly identified current flaws in the tools, and the community tries to lead you to the tools that are known to work well in spite of the issues. We need to work a little bit more on the other tools to make them as good, but large parts of the community are just not the right people to work on that problem. It's not a design problem, it's fixable, but you just have more people who know how to make new differential equation solvers than you have people who can compile a binary, so the tooling ecosystem moves at a different pace from the scientific packages. This is changing with the help of some commercialization aspects though, but it is something we have to be cognizant of.
The world is bigger than you.
There is no need to insult people around.
I totally get that there are workflows that Julia is inappropriate for, but when someone says that a tool shouldn't be released because it doesn't target their workflow, I really don't know what to call that other than blindingly self centred.
More seriously though, just because some language may be considered "general purpose" doesn't mean it's the best language for everything, or should replace other languages outside of whichever particular applications where it excels.
No I don't. It needs some work in some places.
Look, everyone wants static compilation, and it's almost certainly going to happen eventually. It's not unreasonable at all to want that. The only thing that would be unreasonable is to (apparently repeatedly!) imply or assert that this lack of static compilation (and/or better caching) is due to some bizarre perversion on the part of the community.
If you want it to happen faster, then stop complaining about it on HN, get familiar with the current attempts at true static compilation in e.g. https://github.com/tshort/StaticCompiler.jl/pull/46 and start contributing yourself.
I like the "AOT-JIT" approach of Julia, and I think a separate compilation step is quite a stupid idea in the first place. I see it as some historical relic from computers with extremely limited memory (by current standards) and outdated business models that want obfuscated binaries.
My main gripe is just that I have to do it every time I run anything if I'm not using a REPL/Notebook, which drive me insane with their inconsistent and opaque global state. This could be solved mostly by caching the compilation results and compiling new stuff incrementally. But I'm starting to lose any hope that this is gonna actually happen.
Some lesser stuff follows from lack of this, but they should be relatively easily fixed.
For what it's worth, I've seen some nasty (though well meaning) pile-ons in the julia discourse, but for the most part it's a pretty nice place. I find though that the Zulip [2] and Slack communities are generally more friendly and relaxes places though.
1) I use VScode and have had 0 problems with Revise. It Just Works when using the Julia extension + built in REPL. I actually prefer the Julia environment to Python in VScode, I have way fewer problems when doing a Notebook-like workflow where I’m writing library code at the same time
2) Agreed, I also wish there was a better story here. I’m constantly frustrated by how little static support there is. It’s getting better though, eg precompilation in 1.6.
3) I haven’t had any issues here outside of the occasional update to VScode Julia extension that botches things
4) I’ve had quite a lot of luck on discourse and GitHub issues, as well as slack for the occasional small question
Rust is great for use cases where you have specific resource management needs, although seems an odd choice for general purpose scientific computing.
Try changing a struct.
Until then, consider this work-around: https://github.com/BeastyBlacksmith/ProtoStructs.jl
Not sure if you can generalize it to "Julia community isn't interested".
If we have more ppl liking Julia and using it for general purpose computing (which it is capable of), then more of these tools will become mature overtime.
Also, Julia is not in the same niche as Rust though, it's more like a Python, so not sure if the comparison is apt.
5. The module system is very primitive.
6. The testing framework is extremely barebones.
I agree with your assessment, Julia is great for crunching numbers etc., but I wouldn't write a whole application in it.
[1]: I actually tried to automate some of this in a project: https://gitlab.com/pfrasa/morcrypto/-/blob/master/test/runte...
In general though, specific issues with use-cases (or even better PRs) are the best way to direct change in an open source language. It's much easier to prioritize development if we know that a specific feature is wanted.
You know, maybe you're right and I can't prove you wrong, but this is the kind of thing that IMHO application developers read and think "yeah no, been there, done that, never again". Because many people's experience is that names do clash, again, and again, and again, and I fail to see why multimethods should solve that.
[1] https://docs.julialang.org/en/v1/manual/style-guide/#Avoid-t...
https://clojure.org/about/spec
You can compare and contrast MLDatasets.jl (uses submodule per dataset) vs CorpusLoaders.jl (uses a extra type argument per dataset)
It is supposed to land in the release after 4.13, which is the next one.
Regarding the scientific computations library there is Owl[1][2] which now has an almost finished book[3].
Things that kind of suck about using Julia for production:
1. Never could get Revise to work, had to restart my REPL everytime I changed any code. Even though Julia 1.6 was a lot faster than 1.5, it still took too long.
2. Couldn't find a static type checker that actually worked (I tried JET and StaticLint). I feel like static typing is just so important for a production system, but of course the community isn't really interested because of the research focus.
3. Editor tooling. The LSP server absolutely sucks. I first tried using it with emacs (both lsp-mode and eglot mode), but it would crash constantly. I think switched to VSCode (much to my chagrin), and that worked marginally better though still very poorly. It was clear that the LSP server had no idea what was going on in my macro heavy code. It couldn't jump to definitions or usages much of the time. It could never correctly determine whether a variable was unused or misspelled either. Coupled with the lack of static type checking, this was extremely frustrating.
4. Never felt like the community could answer any of my questions. If you have some research or stats question, they were great, but anything else, forget about it.
Will all of that being said, I do still use Julia for research and I find it works really well. The language is very nicely designed.
All and all, I decided to ditch Julia and decided to go with Rust (after some consideration of OCaml, but unfortunately the multi-core story still isn't there yet) and am a lot happier.