News for hackers. Stuff that matters.
Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by ViralBShah on Sunday October 11, 2020 @09:22PM (#24750604)
Nobody is spinning anything here. Julia's compilation time is a known quantity. See my comment in https://news.ycombinator.com/item?id=24750559 about the different phases of Julia's compilation and execution (from a user's perspective).

Obviously, if you are not working on a problem that doesn't need Julia's speed (and makes it worth paying the compilation cost), and you are more comfortable with a different tool, you should use that. You may in fact be better off using Python, or shell scripting or even Excel.

Precompilation time in Julia is akin to `make` in a project with C code. You only compile the library once and use it repeatedly, until you update it. Is it a dealbreaker? Perhaps it is for you (assuming you are using precompilation in the right context). It is not for many who work with Julia day in and day out, and it is something that we continue to improve.

Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by fermienrico on Monday October 12, 2020 @12:13AM (#24751324)
Thanks for the insight. I think still for most engineering development - its mostly the one-off launch time that predominates. This is exactly why I don't use C++ for general purpose use.

There is something to be said about how easy Python makes development look - despite of the package management warts.

Btw, is there a reason why packages can't be precompiled and even distributed? I am sure you guys have thought about that.

Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by cbkeller on Monday October 12, 2020 @03:33AM (#24752227)
> precompiled and even distributed

There is work on this. This can be done with https://julialang.github.io/PackageCompiler.jl/dev/, but so far only to a rather large binary / “bundle”

IIUC, In principle it should be possible to do much better if you knew for sure at compile time which methods you would need to dispatch to for the data you ultimately want to run on.

Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by ViralBShah on Monday October 12, 2020 @09:34AM (#24754172)
Yes, you can use PackageCompiler for your collection of packages. We are also exploring doing this in the background when you add more packages.

The reason why we can't distribute precompiled .ji files at a package level is because of the way the package resolver works. It depends on the exact versions of dependencies of every package - and even for the same version of an end-user package, there can be slightly different versions of the dependencies installed depending on constraints imposed by other packages.

One major improvement coming in 1.6 is multi-threaded precompilation, and it will leverage all the cores you have.