News for hackers. Stuff that matters.
Re:The Elements of Differentiable Programming by samatman on Friday March 22, 2024 @10:12PM (#39796860)
You seem somewhat obsessed with the idea that reverse-mode autodiff is not the same technique as forward-mode autodiff. It makes you,,, angry? Seems like such a trivial thing to act a complete fool over.

What's up with that?

Anyway, here's a forward differentiation package with a file that might interest you

https://github.com/JuliaDiff/ForwardDiff.jl/blob/master/src/...

Re:The Elements of Differentiable Programming by patrick451 on Friday March 22, 2024 @11:21PM (#39797135)
I can't reply to the guy saying julia is the only one. But there are others.

Ceres uses dual numbers

https://github.com/ceres-solver/ceres-solver/blob/master/inc...

This library from google is used everywhere in robotics, so it's hardly some backwater little side project.

So does c++ autodiff https://github.com/autodiff/autodiff/blob/main/autodiff/forw...

So does Eigen: https://eigen.tuxfamily.org/dox/unsupported/AutoDiffScalar_8...

Re:The Elements of Differentiable Programming by MikeBattaglia on Friday March 22, 2024 @11:38PM (#39797200)
Re:The Elements of Differentiable Programming by fpgamlirfanboy on Friday March 22, 2024 @10:15PM (#39796881)
it's amazing to me that pointing out a straight up mathematically factual inaccuracy is considered "angry" and "acting a fool".
Re:The Elements of Differentiable Programming by samatman on Friday March 22, 2024 @10:25PM (#39796912)
> there's a reason no one uses dual numbers (non-standard analysis) for anything (neither autodiff nor calculus itself)

wrong

> there are no [dual] numbers anywhere in [...] any other serious autodiff implementation

wrong

> please show me the source line number

did

> i'm wrong and this other guy is right

correct.

> this is why i hate this kind of "TIL, gee whiz" math tidbits

acting a fool. angrily so.

Re:The Elements of Differentiable Programming by fpgamlirfanboy on Friday March 22, 2024 @10:42PM (#39796978)
is this like some kind of version of truman show? the full sentence is

>please show me the source line number in pytorch or tensorflow that defines this number

why? because the original comment makes a claim about pytorch. it's all right there in black and white.

Re:The Elements of Differentiable Programming by samatman on Friday March 22, 2024 @10:53PM (#39797022)
Which claim, this one?

> Most autodiff packages (such as Pytorch) use something not much more advanced than this, although there are optimizations to speed it up (e.g. reverse mode diff).

We get it: you're a mechanic who thinks that makes you an automotive engineer.

Re:The Elements of Differentiable Programming by fpgamlirfanboy on Friday March 22, 2024 @10:55PM (#39797038)
i haven't a clue what that means but when julia's autodiff is the only one that is implemented using dual numbers i think the claim is pretty obviously false and seemingly everyone has an issue admitting that. without a doubt you'll just say something else sarcastic rather than admit it.