News for hackers. Stuff that matters.
Re:Nilang.jl – A Reversible Julia DSL by dlivingston on Sunday October 11, 2020 @07:26AM (#24745064)
Forgive my ignorance, but what applications would this be used in where just maintaining copies of the initial parameters to f() wouldn’t work?
Re:Nilang.jl – A Reversible Julia DSL by sriku on Sunday October 11, 2020 @10:34AM (#24746076)
Very interesting. Not read the paper yet, but plan to.

Reversibility is important in quantum computing. Quantum circuits must transform input to output in a "unitary" manner, which is reversible. If you consider the input to output as a linear transformation matrix (with complex values), then the complex conjugate of the matrix gives the "inverse function".

This is probably useless info, but reversibility in the classical sense is also interesting due to the energy bounds of computation. The Landauer limit (kTln2) [1] gives a lower bound of energy that must be dissipated to destroy one bit of information in a computation. A reversible calculation does not destroy bits.

[1] https://en.wikipedia.org/wiki/Landauer%27s_principle

Re:Nilang.jl – A Reversible Julia DSL by cbkeller on Monday October 12, 2020 @12:15AM (#24751330)
I was wondering about this latter point as well, and apparently this is exactly the goal if one could implement reversible computing at the hardware level [1]:

> One of the earliest proposals to reduce heat from lost bits was to make computer circuits reversible. A reversible circuit has exactly as many outputs as inputs, assigning one output pattern to every input pattern and vice versa. That means each input can be reconstructed from the output; no bits are lost, so reversible circuits will not give off heat from bit loss. Furthermore, reversible circuits can simulate standard logic functions and can therefore be used in any computer.

[1] https://www.americanscientist.org/article/computers-that-can...

Re:Nilang.jl – A Reversible Julia DSL by sinenomine on Sunday October 11, 2020 @02:10PM (#24747767)
One can use this to express a large part of a deep ML model as a reversible function, thus drastically lowering memory requirements for backpropagation. The same technique, but applied by hand was recently used to speed up training for transformer language model: https://arxiv.org/abs/2001.04451

I think in the future we will see a trend of expressing most of DL model as reversible computation, with minimal irreversible module in the end and in the beginning.

Re:Nilang.jl – A Reversible Julia DSL by amitport on Sunday October 11, 2020 @08:50AM (#24745433)
I think the space requirements are different.

Even if you only care about previous applications of f. Keeping snapshots of every input isn't always feasible.

Re:Nilang.jl – A Reversible Julia DSL by GiggleLiu on Sunday October 11, 2020 @03:33PM (#24748394)
Snapshots can be freed through reverse computing in NiLang. Reverse computing has subtle relationship with checkpointing, see author's blog: https://nextjournal.com/giggle/reverse-checkpointing .
Re:Nilang.jl – A Reversible Julia DSL by e12e on Sunday October 11, 2020 @09:51AM (#24745753)
I think the core idea is that it actually gives you an inverse function, based on your regular (in the DSL) function definition.

There's as far as I can tell no inherent need to call the regular function first - given any function f, you get a callable inverse function ~f.