Can We Solve Linear Algebra Problems at Extreme Scale and Low Precisions?

Re:Can We Solve Linear Algebra Problems at Extreme Scale and... by thxg on Friday September 17, 2021 @03:01AM (#28562152)

The article covers ways to obtain smaller error bounds, mainly for testing algorithm stability in practice.

The methods proposed are very varied in nature: blocked algorithms, x87 80-bit arithmetic, AVX2/ARMv8-A fused multiply-add (FMA) operations, and probabilistic bounds.

While the article is otherwise very good, I am unsure about one of those in 2021: 80-bit arithmetic. The corresponding x87 instructions are deprecated by Intel, and their performance is disastrous. For example, the complex algorithms implemented in software by glibc to compute transcendental functions (sin, cos, exp, ...) are faster and more accurate than their single instruction x87 FPU counterparts (FSIN, FCOS, ...)!

I would be very curious to see, in a few algorithms, a comparison of x87 arithmetic with double-doubles and even low-precision MPFR. MPFR, by the way, may have its place here if we're talking about sacrificing some performance for higher precision.

[ Parent | on HN ]

Re:Can We Solve Linear Algebra Problems at Extreme Scale and... by gpderetta on Friday September 17, 2021 @05:37AM (#28563149)

The trascendental instructions are slow and have low accuracy, but the "normal" x87 fp ops are still fast (except for the lack of vectorization).

[ Parent | on HN ]

Re:Can We Solve Linear Algebra Problems at Extreme Scale and... by adgjlsfhk1 on Friday September 17, 2021 @10:26AM (#28565867)

Yeah. The problem with them is that using double-double based methods gives you 104 bits of precision that works with vectorization (and therefore is faster than x87).

[ Parent | on HN ]

Re:Can We Solve Linear Algebra Problems at Extreme Scale and... by gpderetta on Friday September 17, 2021 @10:44AM (#28566043)

I assume double-double would be the emulated __float128 type in GCC (or hand written equivalent)? Can GCC vectorize it or it needs to be done by hand?

[ Parent | on HN ]

Re:Can We Solve Linear Algebra Problems at Extreme Scale and... by jcranmer on Friday September 17, 2021 @11:47AM (#28566851)

__float128 is emulated quad-precision, I think.

[ Parent | on HN ]

Re:Can We Solve Linear Algebra Problems at Extreme Scale and... by gpderetta on Friday September 17, 2021 @12:04PM (#28567080)

Not exactly, the mantissa is increased, but the exponent is the same as standard double precision, so it is not an IEEE quad.

[ Parent | on HN ]

Re:Can We Solve Linear Algebra Problems at Extreme Scale and... by thxg on Friday September 17, 2021 @09:46AM (#28565424)

You're right, I didn't know that. In a quick "long double" vs "double" microbenchmark, I see a negligible penalty for additions and comparisons, and 37% penalty when multiplications are involved.

[ Parent | on HN ]

Re:Can We Solve Linear Algebra Problems at Extreme Scale and... by snovv_crash on Friday September 17, 2021 @05:40AM (#28563170)

Agreed, I've seen better results from algorithmic approaches.

For example, Takahashi for bounded-error on extremely large dot product. Depending on use case even doing a pseudo-Takahashi where you only merge the error back once every N iterations can unblock instruction-level parallelism letting the performance hit be only around 2x relative to naive dot products, but with still effectively the same guarantees.

[ Parent | on HN ]

Re:Can We Solve Linear Algebra Problems at Extreme Scale and... by evancox100 on Friday September 17, 2021 @09:25AM (#28565200)

I think they were only recommending x87 for a final summation

[ Parent | on HN ]