Re:Parallel Supercomputing for Astronomy with Julia
by KenoFischer on Thursday October 11, 2018 @03:45PM
(#18196464)
The JIT compiler was active, but most of the computationally heavy parts were ahead of time compiled. The compiler had a number of tweaks to increase the quality of the generated code (though at the expense of a good amount of compile time). Generating the ahead of time image probably took a good 5-10 minutes (much more if we had tried to do it on the KNL nodes that we actually ran on), so it was infeasible and would have been a waste of time to try to do so in a JITted fashion. Some JIT compilation probably still happened (for code paths that weren't in the pre-compilation trace, e.g. logging code or exception code paths hit by a few processes).
Re:Parallel Supercomputing for Astronomy with Julia
by noobermin on Thursday October 11, 2018 @04:08PM
(#18196661)
For some reason, my brain forgot you did this on KNL. I guess even with cross compilation and then passing it to the platform compilers and it performing well is a sign it's good. Anyway, it's great you guys got this through especially with a tool like Julia. At least in my corner of academia, people utilizing "untested tools" unnecessarily raises the ire and skepticism of peers (for rather stupid reasons). I appreciate you guys "making the first move" which makes it easier for the rest of us looking to branch out and use these tools for our own codes.
Re:Parallel Supercomputing for Astronomy with Julia
by KenoFischer on Thursday October 11, 2018 @04:14PM
(#18196702)
Just wanted to clarify that all code generation was done through LLVM (as opposed to Intel's proprietary compilers).
Did you use one of the compilation options for this or just the JIT compilation (if direct compilation is available, I'll be honest, I'm not that abreast of Julia developments). One of the key things for more computational simulation tools (as opposed to just analysis) is we find that the awesome compilers for C and Fortran codes are a huge boon, especially on systems that have ones that improve on stock intel (not to mention gcc) by a good factor. You lose that if you're just using the jit for the sake of using a "nicer" language, I guess.
I had similar sentiments to the other cat who asked about the 15 minute run, but I guess it's fun to show it "can be done." I was planning to play a little in Julia and this shows it can be worth it.