News for hackers. Stuff that matters.
Re:Implementing the Exponential Function by dvt on Sunday June 28, 2020 @04:29PM (#23671790)
This is an awesome article! One of my favorite SO answers I remember researching dealt with the Padé approximation of tanh[1] (which I found was significantly better than the Taylor expansion), but the caveat was that it only worked within a very narrow neighborhood.

I will say that the article didn't really touch on techniques that minimize IEEE 754 subnormal[2] performance impact, which is a very interesting problem in itself. A lot of real-life implementations of something like e^x will have various clever ways to avoid subnormals.

[1] https://stackoverflow.com/a/6118100/243613

[2] https://mathworld.wolfram.com/SubnormalNumber.html

Re:Implementing the Exponential Function by fractionalhare on Sunday June 28, 2020 @05:30PM (#23672215)
Hi, I am the author. You make a good point about subnormals, thank you. I will jot that down for a future update to the doc :)

It is funny you mention the Padé approximation; I was debating whether or not to include that in this article. Originally I planned to stop after Minimax (using Remez), but if I include rational approximation schemes anyway (e.g. Carathéodory-Fejer), it probably makes sense to implement and analyze that as well.

Re:Implementing the Exponential Function by dia80 on Sunday June 28, 2020 @06:46PM (#23672672)
Just jumping into to say I am loving the article, thanks! I'm dealing with a lot of exps in an optimization problem right now and frankly only have cursory understanding of what's going on under the hood so this was very enlightening for me. Thanks again.
Re:Implementing the Exponential Function by raphlinus on Sunday June 28, 2020 @09:27PM (#23673520)
You might enjoy a few of my favorite sigmoids: https://raphlinus.github.io/audio/2018/09/05/sigmoid.html

That's not a deep exploration of the numerical analysis (especially compared with OP), but it does make one point: traditional "numerical analysis" techniques tend to assume the traditional four mathematical operators as canonical, but if your goal is to get something computed on actual computer hardware quickly and reasonably accurately, you have a wider palette to play with. In particular, inverse square root is a primitive about as efficient as reciprocal on modern hardware (or if using Carmack's wtf trick).

Re:Implementing the Exponential Function by dvt on Sunday June 28, 2020 @11:08PM (#23673995)
This is great! Sometimes I wish I took more analysis back in school.
Re:Implementing the Exponential Function by m4r35n357 on Sunday June 28, 2020 @05:06PM (#23672057)
Just a suggestion, have you looked at the MPFR project, and derivatives: https://www.mpfr.org/

http://mpfr.loria.fr/mpfr-current/mpfr.html#index-mpfr_005fs...

It is way to deep in the grubby details for my pay grade, but who knows?