News for hackers. Stuff that matters.
Re:Native Clojure with GraalVM by wink on Tuesday September 18, 2018 @04:25AM (#18013478)
After hearing about GraalVM first time at a PolyConf talk last year I sat down with one of the GraalVM folks and we (well, I was just providing info) did a few performance measurements if this could speed up the startup time (especially for leiningen tasks) - the preliminary results were pretty cool.

So if you're using Clojure and Leiningen and are annoyed by the long startup times... there's potential :)

Re:Native Clojure with GraalVM by zcam on Tuesday September 18, 2018 @05:02AM (#18013622)
It's not all unicorns though.

Performance can vary quite a lot from task to task, also compile times suffer a lot and you lose some dynamic features.

So from what I understand it might be useful for clojure cli apps, but at this point I am not sure it brings a lot, but this is early days.

The cool part is the polyglot capabilities imho.

Then there's the elephant in the room: the licensing and potential changes in the long run.

For clojure if you want decent cli/startup experience there's always cljs, lumo or you can go with something like fennel-lang, a clojure looking lisp that "transpiles" to lua(jit), so 0 overhead compared to lua and super fast startup.

https://fennel-lang.org/

Re:Native Clojure with GraalVM by akerro on Tuesday September 18, 2018 @08:19AM (#18014568)
>Performance can vary quite a lot from task to task, also compile times suffer a lot and you lose some dynamic features.

That's actually really bad in Graal. I compiled a few classes project in Java, JS and Ruby. Compilation time was measured in minutes and memory usage most of the time was >8GB.

Re:Native Clojure with GraalVM by aseipp on Tuesday September 18, 2018 @12:56PM (#18016948)
To be clear (for anyone reading), it's SubstrateVM's whole-program compiler (and corresponding analysis') that does this, not Graal itself -- Graal can be used as an ordinary JVMCI compiler in an ordinary JDK, like HotSpot C2. Graal is really just a very fancy compiler-backend-as-an-API that can emit native code from a CFG (like LLVM, sort of), just written in Java. SubstrateVM uses Graal to emit binary object code from the JVM class files you specify at compile time, when you use the 'native-image' tool -- but it otherwise has a different compilation pipeline entirely (AFAIU.) So Graal can both be used for a JIT or AOT compiler...

SubstrateVM was designed more to make Graal/Truffle based interpreters like TruffleRuby and FastR have fast startup time with small binaries, which is a large and complex project, more than it was meant to compile small, random one-off Java/JVM applications for CLI usage. From what I remember in my few random experiments the compile time isn't quite linear in size at least (e.g. compiling a 10 line Java class file takes 2 minutes, but compiling a 100 line one takes only marginally longer, comparatively.)

(Using SubstrateVM also means that there is no JIT for the JVM class files you compile to native code, so you're dealing with a purely ahead-of-time compilation scheme. Graal's online JIT still kicks in for SubstrateVM-based interpreters like TruffleRuby, however. This is an important distinction to understand.)

Hopefully they can improve the compile time requirements a bit for cases like this, since it's surely more popular and common case that people are looking for, as opposed to writing Truffle-based interpreters.

Re:Native Clojure with GraalVM by wink on Tuesday September 18, 2018 @05:08AM (#18013653)
Yeah, but I don't see anyone rewriting leiningen in cljs anytime soon.

For general CLI tools, yes, I'd also look into cljs at this time.

Re:Native Clojure with GraalVM by tosh on Tuesday September 18, 2018 @06:16AM (#18013949)
The CLI example in the talk is quite impressive

edit: a bit more on zprint & GraalVM and a comparison to the equivalent code on node:

  graalVM native startup time: 0.055s
  node startup time: 0.260s
execution time is also more than 1 order of magnitude faster w/ graalVM native compared to node

https://github.com/kkinnear/zprint/blob/master/doc/graalvm.m...

Re:Native Clojure with GraalVM by zcam on Tuesday September 18, 2018 @05:18AM (#18013707)
You'd be surprised, there are people in the community with some clever ideas to do that with a (fast)client/server model and change diffing (incremental builds), but it's super early days.

With tools.deps and the need of some companies for more advanced build setup a lot of people are actually thinking about these issues.

Re:Native Clojure with GraalVM by wink on Tuesday September 18, 2018 @07:28AM (#18014273)
I'm not denying that someone, somewhere is working on something.

But I'd be surprised indeed, I'm not watching the whole clojure space closely, I can only say that no one hit the #leiningen irc channel or posted something on the bugtracker/proposed a PR yet.

And the client/server model.. sure, nailgun and grenchman were a bit clunky, but none of these efforts have gained widespread adoption, afaik. Most people don't just go out of their way to tweak their build tool.

Re:Native Clojure with GraalVM by zcam on Tuesday September 18, 2018 @08:09AM (#18014501)
> And the client/server model.. sure, nailgun and grenchman were a bit clunky, but none of these efforts have gained widespread adoption, afaik. Most people don't just go out of their way to tweak their build tool.

I was not thinking about these two, nor was I thinking about a patch for boot/lein.

That said the slow startup issue is only really valid for cli apps (and maybe CI), if your dev workflow relies on re-starting your repl all the time, it's a broken workflow.

Re:Native Clojure with GraalVM by jjwiseman on Tuesday September 18, 2018 @03:30PM (#18018378)
There is a group of people that love to say this ("If you're not using a repl or restarting your repl all the time, you're doing it wrong"), but I've been using Lisp for more than 20 years and I don't agree. A repl is nice, and I use one often, but it does have tradeoffs. One is persistent state--It can be quite easy to get your repl into a state where your code works, but if you run the app from scratch it doesn't because definitions are out of sync (e.g. you renamed a function, but forgot to change the name at a call site, so you end up calling an old version of the function.)

Developing in a repl is nice, but it's not the only allowed or legitimate or even good way to do development in Lisp or other dynamic, interactive languages. Clojure has an issue with slow startup (another tradeoff made for good reasons that don't fit everyone's situation)--denigrating other people's development approaches doesn't solve that, it comes off as defensive.

Re:Native Clojure with GraalVM by kazinator on Tuesday September 18, 2018 @03:59PM (#18018598)
> I've been using Lisp for more than 20 years and I don't agree. A repl is nice, and I use one often, but it does have tradeoffs.

Me neither, and I also carry decent Lisp cred.

I know what I'm doing to the extent that I can write a bunch of code in a file, and have it work, more or less.

If I experiment with changes to code, I want to see the diff (as in git diff).

Actually developing in a REPL seems very scatter-brained; you don't know what you're running since you've been mutating things left and right.

Those that develop in a REPL actually develop in some kind of text editor which sends expressions to the REPL; that's what allows them to save the code properly to a file. Problem is, you now have three versions of the code to keep straight: editor, file and image.

Okay, so you have massaged things into working. Well, which is the code that is working? Obviously, the code that is in the image. But of the textual codes does it correspond to? If you just save the content of every edit buffer to disk, is the code on disk exactly that code that is working in the image?

Even if I have to work with a running image (say there is a lot of state that is difficult to reproduce from a clean start), I'm still going to work with some files, which I will edit and completely save to disk, and load all together as a unit into the running image, instead of evaluating individual expressions out of an edit buffer.

Re:Native Clojure with GraalVM by zebraman on Tuesday September 18, 2018 @05:08AM (#18013656)
For command-line applications, it opens up an interesting space. There was a post on HN a while ago: https://www.astrecipes.net/blog/2018/07/20/cmd-line-apps-wit... though the system requirements for compilation are at the moment very onerous.
Re:Native Clojure with GraalVM by wink on Tuesday September 18, 2018 @05:12AM (#18013680)
Apparently I missed this one, thanks!

> So you have no excuses not to build your CLI tools in Clojure now.

Well I do disagree with this part, see other sibling comments (not by me). I'm not even sure GraalVM is really production-ready, notwithstanding the potential licensing and long-term issues.

I do agree it's at least worth trying.

Re:Native Clojure with GraalVM by zebraman on Tuesday September 18, 2018 @05:32AM (#18013765)
We are starting to do that, and for very simple operations, it works (like scripts for maintenance that we deploy remotely, and run as cron jobs). So far no big surprises, but we were very careful.

The big minefield is that you do not really know which libraries will go boom and where, as any place could make use of class loading.

Re:Native Clojure with GraalVM by repolfx on Tuesday September 18, 2018 @01:24PM (#18017247)
What licensing issues do you have in mind. Graal and SubstrateVM are all GPL-with-classpath-exception licensed, same as Java itself. It's an open source license.
Re:Native Clojure with GraalVM by Rovanion on Tuesday September 18, 2018 @04:36AM (#18013509)
Did Leiningen compile through Graal without modification?
Re:Native Clojure with GraalVM by wink on Tuesday September 18, 2018 @05:00AM (#18013619)
No, at least not back then. Otherwise we might have an official branch or PR already. I didn't really follow up on this, this was just a quick hack session to see if we could get something like "lein version" or "lein deps" or something to run and compare the speed.

One main problem I also see is the distribution. Currently leiningen only needs "some sort of java executable" and then the rest is pulled in. So would you have to have GraalVM installed or could this also be bootstrapped.