Julia: Dynamism and Performance Reconciled by Design (2018)

Re:Julia: Dynamism and Performance Reconciled by Design (2018) by attractivechaos on Monday October 26, 2020 @06:25PM (#24902240)

I am not sure faster compilation is the way to go. Cutting 30 sec to 15 helps little and the acceleration of compilation can't catch up with the growth of packages in the long run. Also, multi-threaded precompilation sounds a bad idea on managed clusters where resource usage should be controlled. I wonder how much Julia devs have done on caching precompiled binaries. That, though non-trivial to implement properly, is the ultimate solution to slow compilation.

[ Parent | on HN ]

Re:Julia: Dynamism and Performance Reconciled by Design (2018) by fanf2 on Monday October 26, 2020 @06:49PM (#24902426)

The manual says “Large modules can take several seconds to load because executing all of the statements in a module often involves compiling a large amount of code. Julia creates precompiled caches of the module to reduce this time.”

https://docs.julialang.org/en/v1/manual/modules/#Module-init...

[ Parent | on HN ]

Re:Julia: Dynamism and Performance Reconciled by Design (2018) by celrod on Monday October 26, 2020 @11:18PM (#24904192)

Just to clarify, those precompiled caches are (currently) not of binaries. At the very least, LLVM still needs to generate code after loading, and often everything up to inference still needs to be run as well.

Julia 1.6 involved a lot of work reducing invalidations, where old methods are invalidated by new definitions and thus forced to recompile. This has the obvious benefit of reducing the total amount of compilation needed, but the secondary benefit of making compiling to binaries more profitable; it's less likely that work will be wasted when someone loads another package.

[ Parent | on HN ]

Re:Julia: Dynamism and Performance Reconciled by Design (2018) by attractivechaos on Monday October 26, 2020 @07:29PM (#24902745)

This sounds great in theory, but in practice julia-1.5.2 still takes 10 sec to start when I use a medium-sized package including its dependencies (16 compiled files in total). If julia is loading cached files, I would expect something below 1 sec.

[ Parent | on HN ]

Re:Julia: Dynamism and Performance Reconciled by Design (2018) by roywiggins on Monday October 26, 2020 @07:54PM (#24902986)

You can manually precompile a system image with libraries you find yourself using often, like Plots.jl. The usability of it stinks, but... you can do it, and it speeds common things up.

https://julialang.github.io/PackageCompiler.jl/dev/examples/...

[ Parent | on HN ]

Re:Julia: Dynamism and Performance Reconciled by Design (2018) by ChrisRackauckas on Monday October 26, 2020 @08:35PM (#24903284)

Its usability is being integrated into the package manager so that it automatically gives warnings and triggers when you need to redo the sysimage. Not in v1.6, but there's been a lot of discussions for doing it shortly after. That is something I'm excited for.

[ Parent | on HN ]

Re:Julia: Dynamism and Performance Reconciled by Design (2018) by attractivechaos on Monday October 26, 2020 @09:20PM (#24903582)

Thanks for the tip. This does look promising.

[ Parent | on HN ]

Re:Julia: Dynamism and Performance Reconciled by Design (2018) by ChrisRackauckas on Monday October 26, 2020 @06:35PM (#24902323)

We recently acquired new hardware and plan to look into it. Invalidations can get in the way, but there are some things that can be done.

[ Parent | on HN ]