News for hackers. Stuff that matters.
Re:Nilang.jl – A Reversible Julia DSL by vsskanth on Sunday October 11, 2020 @12:14PM (#24746784)
How does this compare to zygote ?
Re:Nilang.jl – A Reversible Julia DSL by GiggleLiu on Sunday October 11, 2020 @03:26PM (#24748331)
1. Scalar level

Various benchmarks (including those in the paper) show NiLang is much better than Zygote to differentiate scalar functions. And Zygote is much faster than TF and PyTorch.

2. Tensor level

Zygote, TF and PyTorch are much better than NiLang, because NiLang's matrix multiplication is not fully optimized, it is much slower than BLAS. (One can wrap BLAS into NiLang, but that does not measure NiLang's programming language level AD performance anymore)

Re:Nilang.jl – A Reversible Julia DSL by cbkeller on Sunday October 11, 2020 @03:12PM (#24748231)
As far as I understand it (which is not very far):

Zygote calculates derivatives using source-to-source automatic differentiation.

This calculates function inverses (so to stretch the analogy a bit, it's kinda like "source-to-source automatic inversion")

Re:Nilang.jl – A Reversible Julia DSL by ChrisRackauckas on Sunday October 11, 2020 @09:28PM (#24750644)
Not necessarily: you can use it to build fast inverse of scalar functions. Here's an example on a vjp of an ODE f:

https://gist.github.com/GiggleLiu/0c4608f70bda050f59992f5fc0...

Re:Nilang.jl – A Reversible Julia DSL by cbkeller on Sunday October 11, 2020 @11:11PM (#24751083)
I'm probably just missing something, but did you mean to say "fast derivatives of scalar functions" by any chance? I.e., that you can use the free inverses to get at the derivatives in some way?
Re:Nilang.jl – A Reversible Julia DSL by ChrisRackauckas on Sunday October 11, 2020 @11:54PM (#24751237)
oh yes, I meant derivatives.