tiarkrompf

Great question! Unary operators are really simple to add: you just look for the operator symbol first thing at the right level of precedence. Same idea as "if (peek == '(') ..." for parentheses, but outside the code that deals with '*' and '^' (if you want those to bind more strongly).

I just pushed a fix for Chrome (apparently it doesn't report affected selection ranges for input events). Hope it works now!

Thanks - very glad to hear. RSS should be doable

Author here - happy to answer questions, as always.

Thanks also for feedback on the format of the article. It's a bit of an experiment on how to present dense information effectively. Some more rationale here: https://tiarkrompf.github.io/notes/?/octopus-notes/

For starters, what if you want to differentiate through code that traverses a collection, for example:

    val x = ...
    ys.map(y => y + x)

Each loop iteration needs to contribute a gradient update to x, which is defined in an outer scope. And what if y => y + x is not given as an inline lambda, but defined elsewhere. It doesn't seem like your blog posts discusses any such cases.

Hi Noel, can you clarify how this corresponds to reverse mode AD? To me it seems more like a delayed or on-demand version of forward AD.

Co-author here - nice to see this discussed and happy to answer questions

Thanks! Fixed now.

Thanks! Yes, it wasn't obvious at all when we started looking at AD either.

Lantern supports a good deal of PyTorch (via Snek, our Python front-end similar to AutoGraph) and can also read ONNX. Full feature parity is not our main goal--so far, supported features have been driven mostly by what is required for certain interesting models.

It's different -- instead of returning a closure, we take a closure as additional parameter (check paper for details). This means that the call stack stays intact and all intermediate values can be stack-allocated.

We also have a more recent and slightly longer draft with additional explanations and GPU training results for ResNet 50 and DeepSpeech2:

"Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator" https://www.cs.purdue.edu/homes/rompf/papers/wang-preprint20...

The Lantern framework is available here: https://github.com/feiwang3311/Lantern

Co-author here. Happy to answer any questions, as usual ...

No surprise here, as Martin was my PhD advisor :)

Hi all,

Author here, great to see our paper on the front page today. Happy to answer any questions!

Since Spark was already mentioned in the comments, you might be interesting in the parallel/follow-on work we did applying similar techniques to accelerate Spark:

"Flare: Optimizing Apache Spark for Scale-Up Architectures and Medium-Size Data" (to appear at OSDI, preprint here: https://www.cs.purdue.edu/homes/rompf/papers/essertel-osdi18...)

If you're running Spark in production and are interested in trying it out, please leave your contact data here: https://flaredata.github.io. We're a small team and have a long waitlist, but should have some open slots in our private beta program this Fall.

Cheers, - Tiark

Thank you -- made my day!

Sounds interesting. About the role of CPS: for specialization purposes, CPS can have some benefits by splitting control flow. For example, if your input is "if (c) a else b" and c is a staged (=symbolic) value, then a CPS interpreter can ignore the control-flow join that's implied by the if-then-else, and continue processing longer paths independently. However, this also quickly leads to exponential code blow-up, and hence, we do not use CPS for the core model described in the paper.

I'm one of the authors. Happy to answer questions.

It's a nice piece of code. I like to use it to explain how to turn interpreters into compilers through staging/specialization: https://scala-lms.github.io/tutorials/regex.html

The talk was part of a summer school at EPFL: http://vjovanov.github.io/dsldi-summer-school/

Afaik, everything was videotaped. I'm sure the videos will appear soon.

"precision" JIT: I'm not sure -- can you tell it what to do? More predictable performance model than HotSpot: absolutely.

(author here) Yes, one of our points is that 'register' is much too low-level, because one can't abstract over it. The same holds e.g. for OpenMP directives. In the paper we propose (among other things) to attach such directives to dynamic scopes. This makes it possible to e.g. unroll all innermost loops, including those in function calls, without putting unrolling directives on individual loops.

Author here. Thought I should share the github link as well: https://github.com/TiarkRompf/lancet

Relite author here. I'd love to learn more about your use cases, feel free to get in touch!

Here is a collection of all his papers: http://infoscience.epfl.ch/search?ln=en&p=bagwell