News for hackers. Stuff that matters.
Re:Implementing Cosine in C from Scratch (2020) by dancsi on Tuesday March 29, 2022 @01:36PM (#30845817)
The implementation of the __cos kernel in Musl is actually quite elegant. After reducing the input to the range [-pi/4, pi/4], it just applies the best degree-14 polynomial for approximating the cosine on this interval. It turns out that this suffices for having an error that is less than the machine precision. The coefficients of this polynomial can be computed with the Remez algorithm, but even truncating the Chebyshev expansion is going to yield much better results than any of the methods proposed by the author.
Re:Implementing Cosine in C from Scratch (2020) by potiuper on Tuesday March 29, 2022 @01:39PM (#30845868)
Re:Implementing Cosine in C from Scratch (2020) by mnarayan01 on Tuesday March 29, 2022 @08:47PM (#30850273)
For anyone else confused by this, the control logic described in the comment happens in https://github.com/ifduyue/musl/blob/master/src/math/cos.c
Re:Implementing Cosine in C from Scratch (2020) by llimllib on Tuesday March 29, 2022 @02:29PM (#30846546)
> Input y is the tail of x.

What does "tail" mean in this context?

Re:Implementing Cosine in C from Scratch (2020) by lifthrasiir on Tuesday March 29, 2022 @02:47PM (#30846749)
It is a double-double representation [1], where a logical fp number is represented with a sum of two machine fp numbers x (head, larger) and y (tail, smaller). This effectively doubles the mantissa. In the context of musl this representation is produced from the range reduction process [2].

[1] https://en.wikipedia.org/wiki/Quadruple-precision_floating-p...

[2] https://github.com/ifduyue/musl/blob/6d8a515796270eb6cec8a27...

Re:Implementing Cosine in C from Scratch (2020) by jhgb on Tuesday March 29, 2022 @02:52PM (#30846812)
Does it make sense to use a double-double input when you only have double output? Sine is Lispchitz-limited by 1 so I don't see how this makes a meaningful difference.
Re:Implementing Cosine in C from Scratch (2020) by lifthrasiir on Tuesday March 29, 2022 @03:09PM (#30847009)
The input might be double but the constant pi is not. Let f64(x) be a function from any real number to double, so that an ordinary expression `a + b` actually computes f64(a + b) and so on. Then in general f64(sin(x)) may differ from f64(sin(f64(x mod 2pi))); since you can't directly compute f64(sin(x mod 2pi)), you necessarily need more precision during argument reduction so that f64(sin(x)) = f64(sin(f64timeswhatever(x mod 2pi))).
Re:Implementing Cosine in C from Scratch (2020) by jhgb on Tuesday March 29, 2022 @03:28PM (#30847235)
But am I correct in thinking that is at worst a 0.5 ulp error in this case? The lesser term in double-double can't be more than 0.5 ulp of the greater term and sensitivity of both sine and cosine to an error in the input will not be more than 1.

Also, in case of confusion, I was specifically commenting on the function over the [-pi/4, pi/4] domain in https://github.com/ifduyue/musl/blob/master/src/math/__cos.c , which the comment in https://news.ycombinator.com/item?id=30846546 was presumably about.

Re:Implementing Cosine in C from Scratch (2020) by lifthrasiir on Tuesday March 29, 2022 @03:36PM (#30847338)
Yeah, sine and cosine are not as sensitive (but note that many libms target 1 or 1.5 ulp error for them, so a 0.5 ulp error might still be significant). For tangent however you definitely need more accurate range reduction.
Re:Implementing Cosine in C from Scratch (2020) by adgjlsfhk1 on Tuesday March 29, 2022 @03:31PM (#30847267)
Double rounding can still bite you. You are forced to incur up to half an ulp of error from your polynomial, so taking another half ulp in your reduction can lead to a total error of about 1 ulp.
Re:Implementing Cosine in C from Scratch (2020) by ufo on Tuesday March 29, 2022 @03:55PM (#30847551)
I think the polynomial calculation in the end looks interesting. It doesn't use Horner's rule.
Re:Implementing Cosine in C from Scratch (2020) by lifthrasiir on Tuesday March 29, 2022 @04:50PM (#30848185)
It does use Horner's rule, but splits the expression into two halves in order to exploit instruction-level parallelism.
Re:Implementing Cosine in C from Scratch (2020) by jacobolus on Tuesday March 29, 2022 @05:27PM (#30848606)
Considering the form of both halves is the same, are compilers smart enough to vectorize this code?
Re:Implementing Cosine in C from Scratch (2020) by adgjlsfhk1 on Tuesday March 29, 2022 @07:29PM (#30849661)
I might be wrong but I would think for something like this vectorizing wouldn't save time (since you would have to move data around before and afterwards. The real benefit of this is it lets you run two fma operations in parallel.
Re:Implementing Cosine in C from Scratch (2020) by brandmeyer on Tuesday March 29, 2022 @02:50PM (#30846780)
This has been the standard algorithm used by every libm for decades. Its not special to Musl.
Re:Implementing Cosine in C from Scratch (2020) by enriquto on Tuesday March 29, 2022 @03:54PM (#30847542)
But isn't this code rarely called in practice? I guess on intel architectures the compiler just calls the fsin instruction of the cpu.
Re:Implementing Cosine in C from Scratch (2020) by stephencanon on Tuesday March 29, 2022 @04:13PM (#30847784)
No. FSIN has accuracy issues as sibling mentions, but is also much slower than a good software implementation (it varies with uArch, but 1 result every ~100 cycles is common; even mediocre scalar software implementations can produce a result every twenty cycles).
Re:Implementing Cosine in C from Scratch (2020) by adgjlsfhk1 on Tuesday March 29, 2022 @04:02PM (#30847639)
No. The fsin instruction is inaccurate enough to be useless. It gives 0 correct digits when the output is close to 0.
Re:Implementing Cosine in C from Scratch (2020) by enriquto on Tuesday March 29, 2022 @04:29PM (#30847956)
> 0 correct digits when the output is close to 0

this is an amusing way to describe the precision of sub-normal floating point numbers

Re:Implementing Cosine in C from Scratch (2020) by adgjlsfhk1 on Tuesday March 29, 2022 @05:00PM (#30848290)
It's not just sub-normal numbers. As https://randomascii.wordpress.com/2014/10/09/intel-underesti... shows, fsin only uses 66 bits of pi, which means you have roughly no precision whenever abs(sin(x))<10^-16 which is way bigger than the biggest subnormal (10^-307 or so)
Re:Implementing Cosine in C from Scratch (2020) by ramshorns on Wednesday March 30, 2022 @06:03AM (#30853243)
In that range, just returning x would be way better. Maybe even perfect actually - if x is less than 10^-16, then the error of x^3/6 is less than the machine precision for x.
Re:Implementing Cosine in C from Scratch (2020) by adgjlsfhk1 on Wednesday March 30, 2022 @11:54AM (#30856904)
the error isn't when x is small. it's when sin(x) is small. the problem happens for x near multiples of pi
Re:Implementing Cosine in C from Scratch (2020) by lifthrasiir on Tuesday March 29, 2022 @04:38PM (#30848054)
It is much more amusing if you describe it in ulps; for some inputs the error can reach > 2^90 ulps, more than the mantissa size itself.
Re:Implementing Cosine in C from Scratch (2020) by jhgb on Tuesday March 29, 2022 @04:22PM (#30847868)
FSIN only works on x87 registers which you will rarely use on AMD64 systems -- you really want to use at least scalar SSE2 today (since that is whence you receive your inputs as per typical AMD64 calling conventions anyway). Moving data from SSE registers to the FP stack just to calculate FSIN and then moving it back to SSE will probably kill your performance even if your FSIN implementation is good. If you're vectorizing your computation over 4 double floats or 8 single floats in an AVX register, it gets even worse for FSIN.
Re:Implementing Cosine in C from Scratch (2020) by stephencanon on Tuesday March 29, 2022 @04:48PM (#30848165)
Moving between x87 and xmm registers is actually fairly cheap (it's through memory, so it's not free, but it's also not _that_ bad). FSIN itself is catastrophically slow.
Re:Implementing Cosine in C from Scratch (2020) by jhgb on Tuesday March 29, 2022 @05:24PM (#30848563)
Fair enough, and I imagine there may even be some forwarding going on? There often is when a load follows a store, if I remember correctly. (Of course this will be microarchitecture-dependent.)
Re:Implementing Cosine in C from Scratch (2020) by chrisseaton on Tuesday March 29, 2022 @04:22PM (#30847866)
> I guess on intel architectures the compiler just calls the fsin instruction of the cpu.

Do people do that in practice? It's on the FPU, which is basically legacy emulated these days, and it's inaccurate.

Re:Implementing Cosine in C from Scratch (2020) by enriquto on Tuesday March 29, 2022 @04:27PM (#30847946)
> the FPU, which is basically legacy

you'll pry my long doubles from my cold, dead hands!

Re:Implementing Cosine in C from Scratch (2020) by jhgb on Tuesday March 29, 2022 @04:31PM (#30847979)
The question is if you wouldn't be better served with double-doubles today. You get ~100 bits of mantissa AND you can still vectorize your computations.
Re:Implementing Cosine in C from Scratch (2020) by enriquto on Tuesday March 29, 2022 @04:37PM (#30848037)
Sure. There should be a gcc flag to make "long double" become quadruple precision.

The thing is, my first programming language was x86 assembler and the fpu was the funniest part. Spent weeks as a teenager writing almost pure 8087 code. I have a lot of emotional investment in that tiny rolling stack of extended precision floats.

Re:Implementing Cosine in C from Scratch (2020) by colejohnson66 on Tuesday March 29, 2022 @05:00PM (#30848297)
How is "quad floating point" implemented on x86? Is it software emulation?
Re:Implementing Cosine in C from Scratch (2020) by chrisseaton on Tuesday March 29, 2022 @05:07PM (#30848375)
You're selectively quoting me - I said it's 'legacy emulated'. It's emulated using very long-form microcode, so basically emulated in software. I didn't say it was simply 'legacy'.
Re:Implementing Cosine in C from Scratch (2020) by eyelidlessness on Tuesday March 29, 2022 @08:14PM (#30850036)
I’m completely out of my depth reading through these comments so I don’t have much of value to contribute, but I do think I can gently say I think the selective quoting was harmless, but deliberate to fit the shape of a harmless joke’s setup. I don’t think there was any intent to misrepresent your more salient information.
Re:Implementing Cosine in C from Scratch (2020) by lifthrasiir on Tuesday March 29, 2022 @04:48PM (#30848164)
x87 trig functions can be very inaccurate due to the Intel's original sloppy implementation and subsequent compatibility requirements [1].

[1] http://nighthacks.com/jag/blog/134/index.html

Re:Implementing Cosine in C from Scratch (2020) by jcelerier on Tuesday March 29, 2022 @06:52PM (#30849354)
Wasn't there some blog article a few years ago which showed how glibc's implementation was faster than fsin ?
Re:Implementing Cosine in C from Scratch (2020) by ajross on Tuesday March 29, 2022 @07:20PM (#30849592)
That's got nothing to do with Musl per se, that's just the SunPro code that basically every C library uses. I'm sure the polynomials themselves (there's another one for the "crest" of the sin curve, and still more for log and exp and the rest of the family) date from somewhere in computing pre-history. Optimizing polynomial fits to analytic functions was something you could do on extremely early computers.
Re:Implementing Cosine in C from Scratch (2020) by jeremysalwen on Tuesday March 29, 2022 @10:37PM (#30850957)
Thanks for explaining this. I actually wrote a SIMD implementation of trig functions years ago, using the techniques you describe.

You can check it out: https://github.com/jeremysalwen/vectrig

I compared several different methods of generating polynomials of different sizes for speed and precision (spoilers: taylor series were the worst and minimax polynomials (Remez algorithm) were the best).

Another (surprising) thing which I learned during the project was that the range reduction was just as (if not more) important to the accuracy of the implementation than the polynomial. If you think about it, you will realize that it's actually pretty difficult to quickly and accurately compute the sin of large numbers like 2^50.

I also tried to directly optimize the coefficients for the accuracy of the polynomial on the required range, but that experiment was unsuccessful.

It's all there in the repository, the implementations, notes about the different polynomials used, and the accuracy/speed statistics for the different methods.

Re:Implementing Cosine in C from Scratch (2020) by jhgb on Wednesday March 30, 2022 @08:42AM (#30854334)
> I compared several different methods of generating polynomials of different sizes for speed and precision (spoilers: taylor series were the worst and minimax polynomials (Remez algorithm) were the best).

I would have expected at least an LSQ approximation with a basis of Legendre polynomials thrown into the mix. I got that as a basic homework in my numerics class once, after we've shown to ourselves in the class that [1, x, x², x³...] is not a really good basis to project things onto.

Re:Implementing Cosine in C from Scratch (2020) by bertr4nd on Tuesday March 29, 2022 @09:50PM (#30850657)
Are there libraries/tools that people use to do Remez/Chebyshev/etc. function expansions? I can do a basic Taylor series expansion by hand but I’m out of my depth with more sophisticated techniques.
Re:Implementing Cosine in C from Scratch (2020) by adgjlsfhk1 on Wednesday March 30, 2022 @09:22PM (#30862835)
sollya is the king here
Re:Implementing Cosine in C from Scratch (2020) by bertr4nd on Wednesday March 30, 2022 @09:48PM (#30863016)
Thanks!
Re:Implementing Cosine in C from Scratch (2020) by toolslive on Tuesday March 29, 2022 @04:22PM (#30847876)
Yup. Chebyshev is the way to go (after 30s of consideration)