Much like all development work, language designers are now required to know much more than how to write an effective static compiler, an effective language syntax, and an effective standard library. A working IDE that integrates easily with all mainstream editors, extensions for all mainstream editors, cross-platform compilation for all major environments, a secure and effective dependency manager, a secure and effective build tool, and incremental compilation are all requirements for a language to get off the ground. Honestly, that seems like a big hurdle for one person or even a dedicated team to manage effectively without major funding outside of what is normally present in research grants. Hosting and curation alone for the dependency package system is a costly problem.
Are we as developers painting ourselves into a corner of stagnating pl technology with our expectations that all these things exist in order for us to call a language "good?"
Julia in particular is built to be a practical language so, these points are valuable.
Julia is a language that is half baked for engineering use. Any one who has ever built anything beyond examples and tutorials, something production worthy and used by hundreds of people - they will attest to how immature Julia is.
That's not the fault of the programming language per se... just that it takes 10+ years to get up to speed in terms of tooling and engineering worthiness.
In addition, there are a lot of problems with the language itself some of which are highlighted in the top comment of this thread.
We love Julia for the same reasons others have stated here: It's easy and intuitive to write, making it productive to work with for both our engineers and our mathematicians. It's reflection abilities make performance tuning and optimization easier than any experience I've had before. The ability to fine tune the assembly emitted for your hotspots _matters_, and it's straightforward to do in Julia. And for the rest of the code, being easy to write, easy to read, and close to the math are all a big win.
I've lived through this.
That's less true than it used to be. Julia wouldn't be where it is without LLVM, and things like VS Code's language server protocol do help. But there is still a lot of redoing things for people who write language tools.
It just shows that while IDE support is a factor, it's not the single determining factor.
Ceylon had horrible marketing and advocacy, for starters. And the IDE was Eclipse, at a time that it was getting increasingly unpopular...
The language itself doesn't matter without its ecosystem. This includes tooling, libraries, quality of documentation, the community, and more.
> if you interview 100 engineers who've tried deploying Julia, a vanishingly small % would recommend it if at all.
Popularity of a choice does not correlate with a choice being correct. It is a fallacy to evaluate a particular tool's usefulness for a particular job by how popular that tool is. And doubly so when most of a random sampling of engineers includes virtually no one experienced in using said tool. In discussions of virtually any non-popular programming language today, the vast majority of commenters on the language have absolutely 0 experience using it day in and day out for its intended use case. All you have in that discussion are hammer users. To them everything is a nail. And they universally hate any new tool that doesn't look like a hammer, because they can't take the time to learn the new tool. Why would they? Everything is a nail anyway. This is disappointing, especially here, because Hacker News is the place where the concept of using a non-familiar language was expressed as a super-hack for startups. My original reply expresses the conjecture that eventually, all pls will look the same and nobody will be able to develop a language that will truly provide an increase in quality, decrease development or maintenance costs, or improve the reliability and safety of systems the world runs on, because people who evaluate languages actually evaluate them for the tooling that largely gets bolted on after the fact by nonexperts (the lsp server / vscode crowd). There are three lsp servers for python. Only one of them works with pipenv on Windows. Does that make python bad? If python were introduced today, as it was when it first came out, would it gain popularity with all the expectations we place upon a language? No. And that would be a shame. The same is true of Javascript and Java. C and c++ would go nowhere. About the only language that would have made it is Csharp and Visual Basic and Kotlin, because they were written to SELL IDEs and tools.
Okay but this is a very engineering-specific mindset. I encourage you to broaden your perspective. "Correctness" of a language choice is more than about the language itself. In most situations, it is better to choose a language that scores 9/10 on how well it fits the problem domain than one that scores 10/10, if the 9/10 language will let you solve the problems faster (pre-existing solutions), with less frustration (good docs) and zero need to reinvent the wheel (good libraries). This is especially true if the 9/10 language will also allow you to recruit more easily due to its popularity.
World would have been a bit better place to live.
This complaint is really common with people that see types and expect them to do something like TypeScript. Julia is not a statically typed language. The types are not there for a type checker to check your code for correctness.
Instead the types allow Julia to dispatch your method call to the correct implementation for the type of your variable at runtime. This is Julia's secret sauce, and it's the main reason you'll hear about Julia programmers declaring about how composable Julia packages are. I can "reach into your package", and define the behaviour of your functions on my own custom types and then whenever anyone tries to call the function/method with one of my types, it'll just work. I don't need to author a pull request to your package or futz around with anything that you've wrote.
Novice Julia programmers often come in thinking that the type annotations are there for ensuring correctness, but that's not it at all. In actuality, you want to be as general with your types as you can, and by default most parameters will probably be untyped. If you need certain behaviour for the function, then you should probably annotate it, but it's definitely not required.
I'm guilty of this misstep as well, one of the biggest things I struggled with when I was new was using ::Array everywhere, when what I wanted was ::AbstractArray.
> Include() includes (is this PHP all over again?) a file into the script!
Use Modules. Don't fault Julia for you being a beginner and not reading the necessary parts of the documentation.
Python is not a statically typed language and yet type hinting sentiment seems to have gone from: "Why do you want it? Python isn't statically typed?" to "Well, mypy is a useful optional package" to "Let's just support type hinting in the language itself". Which is to say I think there is value to type checking by development tools even in dynamically typed languages.
--
On a slightly unrelated note, Julia seems to be similar to Python when it comes to types; that is to say both appear to be dynamically and strongly typed. But take that with a grain of salt as I don't have any first-hand knowledge of Julia.
The thing that’s interesting about Julia is that it’s sort of like a verb first language instead noun first. You describe the behaviors that you want by creating methods with one name, getindex, setindex!, etc. that defines the API. Then you can implement that for any type by creating a new method with the same name and a different type signature.
Edit: In fact, I think that over-annotating your methods is probably the number one mistake that new Julia programmers make. Adding a type to the type signature doesn't make your code fast. Adding too many types will restrict your code and it won't work when you think it will because you've told the compiler that this only works with 3-dimensional arrays of type Float64 and you're trying to call it with a 3-dimensional array of type Float32 or Int64.
What makes code fast is type stability, but that's a whole other topic.
That's not a bug, that's a feature. Libraries are abstractions. I get an API and it encapsulates the behavior of the library. That is a nice thing. Being able to modify runtime behavior of internal libraries can lead to insane jumblygoo of code spathetti that would bring the finest programmers to their knees.
I don't want implicit behavior at runtime. I want explicit behavior in case of non-contractual externalities (wrong user input for e.g. at runtime). I want the program to fail so it can be patched.
For machine learning models we have the layer that handles the low level operation (sums, multiplication), which are swappable (you can have an implementation that runs in the CPU - Julia's Base - and an implementation that run in the GPU - CUDA.jl - and even a TPU - XLA.jl or Torch as backend). Above you have the tracker (the layer responsible for the autodifferentiation logic, which includes Tracker, Zygote, ForwardDiff). And above you have the library with rules for generating gradients (DiffRules, ChainRules), and above you have ML constructs (NNLib), and above ML frameworks (Flux, Knet) and above more specialized libraries like DiffEqFlux.
Whoever writes the ML framework doesn't need to care about the backend, whoever writes the GPU backend doesn't need to care about ML framework. This is not because the person writing the GPU backend patched the ML framework, but because the ML framework legitimately doesn't care about how the low level operations are executed, it doesn't work on that level of abstraction. And the user of the ML library can still see it like a monolith not unlike Pytorch or Tensorflow when he imports a library like Flux, until he wants to extend them and then he will find that they are in fact many independent swappable systems that compose into something more than the sum of it's parts.
I'm not sure what you are referring to here. Yes, sometimes after you've installed new packages, the Julia VS Code Language Server has to do some re-indexing which can take some time, but they've improved this and you can still program and use the integrated REPL while this is occurring.
> Ctrl-click navigation does not work to lookup definitations in libraries
In VS Code, F12 (Go to Definition) does work. You can also use `@edit foo(x)` in the REPL.
> Filters and maps are not lazy
Use Iterators.filter or use generator expressions.
> Package management is done as a part of script execution
I'm also not sure what this refers to, but the package manager in Julia is one of the best things about Julia.
> Editors (Juno) do not check types at runtime when the information is readily available
The VS Code Julia extension has a linter that is pretty helpful. It's not perfect, but they're actively working on developing it.
> DataFrames.jl: Transform does nonsense (use map instead)
The new select and transform functions in DataFrames.jl are actually quite powerful and useful.
> DataFrames.jl: No transpose
Transpose is not a generic concept for a table. Sure, it might make sense in specific cases, but in general it doesn't make sense to transpose a table.
From my intuition, I don't see anything stopping the function from existing -- it can be applied to any arbitrary table. You probably don't want to transpose your table, except when you want to, but that's true of any function -- I'm can't imagine any scenario where transpose(table)->table would as an algorithm fail (unless I suppose if julia tables include header rows, in which case there's probably no generally correct definition)
A datatable is (or is isomorphic to and can be analyzed as) a mapping from row numbers to tuples of a given shape[0].
The transpose of table will only be table (a mapping from rows numbers to tuples of a common shape) if the tuples of the starting table were homogenous (every field of the same type.)
This works with, say, matrices where all the elements are numbers; but it fails in the general case.
[0] Yes, I know, Julia defines them as columns of arrays, and columnar organization is ideal for all kinds of processing tasks. For me, thinking about and explaining the the problem with transpose works easier thinking about it in row-oriented form (which is logically equivalent). In column-oriented description, a datatable is an ordered set of columns, each of which is a homogenous array, but if you try to transpose it, each of the columns of the result would be a heterogenous array unless the columns were all of the same type to start with. So, again, it fails to be a table->table function except in the case where the starting table consists of columns of identical type.
Scenario #1: If the tuples are the same shape (type, size), it's fine
[
(string, int, date)
(string, int, date)
(string, int, date)
]
[
(string, string, string)
(int, int, int)
(date, date, date)
]
Scenario #2: Assuming its legal, if the tuples are differently shaped (by datatype), its weird (but that was true of your original table anyways), but you can still do a valid transposition to produce a valid table
[
(string, int, date)
(int, date, string)
]
[
(string, int)
(int, date)
(date, string)
]
Scenario #3: Similarly to scenario 2, If your tuples are differently shaped (by size), you can still do a transposition
[
(string, int)
(string, int, date)
]
[
(string, string)
(int, int)
(date)
]
In the latter two cases, I don't know what you'd want to do with the transposition (or even its input), but I don't see anything stopping the operation itself from being reasonable/consistent/valid.
Is there another scenario I'm failing to imagine?
In the row-oriented view: each row of a datatable is a tuple of the same shape (size and order of types as every other row) -- just like a database table. So, if the shape is (string, int, date) for row #1, its that shape for every row.
In the column-oriented view, each column is a homogenous array: every element in the column has the same type.
> [ (string, int, date) (string, int, date) (string, int, date) ]
Sure, this is a fine starting table; in row-oriented, its shape (the shape of every row) is (string, int, date). In column-oriented view, the table as a whole can be viewed as a tuple of shape (string[3], int[3], date[3]) because it has three rows. Cool.
> transposed:
> [ (string, string, string) (int, int, int) (date, date, date) ]
Right, this is no longer a datatable. The first row has shape (string, string, string). So, if its a table, the other two rows must also have shape (string, string, string); but instead, each has a different shape.
Ok, that makes sense.
Edit: By the Visual Studio paradigm, I mean a paradigm where all the stuff is made around visual interactions with the IDE. For example, showing autocomplete which works because the programmer is very constrained in OOP (foo.b completes to foo.bar) and statically typed languages with Java and C# as prime examples.
I think these constraints are made around the idea that programmers are dumb and that it mostly is a distraction from how one actually wants to think about the problem at hand.
What is this supposed to mean?
> - DataFrames.jl: No transpose
This is actually going to be fixed in the next few days (https://github.com/JuliaData/DataFrames.jl/pull/2447). Given the limited amount of work it required, it sounds quite exaggerated to mention it as a major limitation of the language.
- Arrays start at one
I'm working on a project where indexing naturally runs from -n to n. I kludged the usual index hackery and was so plagued with off-by-one errors that I gave up on it. Then I remembered Tim Holy had a blog post about OffsetArrays.jl and started using that.
Problem solved.
Seriously, the most annoying thing I find about Julia is that the various packages fairly often have "creative" names, that make it difficult discover that they might actually be useful.
YMMV.
Ah, thank you. I've been tinkering around with Julia, and was trying out 2D random walks. Not being able to use a 2 dimensional array and store the position [0, 0] confused me. I ended up starting at X,X (where X > total number of moves) and then got stuck with plotting it nicely, because I wanted the plot axes to show how far it had moved, not X +- how far it had moved.
Here's a recent example of an upgrade, from code with stride calculations to a more abstracted version, which is generic to offsets & number of dimensions:
https://github.com/JuliaLang/julia/pull/37367/files#diff-29c...
I don't really remember if zero-indexing was a big deal when I was learning to program. Are there big advantages for non-programmers? I figure that's why Lua went the 1-index route.
And neither is 1-indexing even though people are always complaining about it. I kinda like when indexing from the third element to the seventh element to be just 3:7, instead of 2:6. and the last element to be length(x) instead of length(x)-1. And that use case is much more frequent to me than creating circular buffers on an array. And in Julia you can just use begin:end as well, or any iterator, it's really overall a non-issue for me.
Not sure about Lua, but in Julia (and Matlab, and R) is just because they actually have vectors/matrices types that have a convention that predates even programming, while it makes sense for languages that an array is basically a memory offset (that maps to a pointer arithmetic logic) to start in 0.
> - Arrays start at one
Oof. Big turnoff for me. For some reason, this particular context switch really grinds my gears. All languages I use (except bash, which grinds my gears) use Dijkstra-style array slices: python, go, c/++, js. I'm sure it makes it easier for Matlab converts.
This, plus the mentioned weaknesses with libs like http, grpc et al, IMHO will relegate Julia to many years of just being wrapped by other languages. To avoid that, I think they should be thinking about continuing to woo more of the engineering crowd - which to their credit I think they've done pretty well so far.
https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/E...
But honestly, it is a royal pain in the ass to use. Mostly because tooling is so crappy (no decent IDE).
For example, I've made these notes on my journey with Julia:
- Juno and VSCode julia is a pain
- Debugging is pain - no watch expressions (but debug> works)
- Works like with matlab (workspaces, designed for the single script use-cases)
- Long registry lookup times (2-5mins) when starting a new session
- No hints from IDE (Juno) on what methods are available for which types
- Ctrl-click navigation does not work to lookup definitations in libraries
- Debugger does not show the full list of elements - there are triple dots in the middle
- Filters and maps are not lazy
- Arrays start at one
- Precompilation times can take 1-2min
- Package management is done as a part of script execution
- Delay (0.5-1s) when processing commands (compilation?)
- Editors (Juno) do not check types at runtime when the information is readily available
- Include() includes (is this PHP all over again?) a file into the script!
- Reuses same REPL between different files
- DataFrames.jl: Transform does nonsense (use map instead)
- DataFrames.jl: No transpose
Obviously, there is a bunch of good things like freakin insane speed and vectorized method calls are awesome, but I struggle seeing myself becoming more producting than Python + Pandas.
It's really a shame that language is strongly and semi-statically typed yet no tooling is making a use of that.