News for hackers. Stuff that matters.
Re:Future instruction set: AVX-512 by sixbrx on Wednesday October 09, 2013 @09:46PM (#6525196)
To amplify this a bit, in binary the relative and absolute errors are within a factor of two, but by a factor of up to 10 for decimal. IBM actually used to use base 16 on some hardware but the rel/abs error disparity of factor 16 hurt and it was eventually abandoned.

There are no comments on this comment yet.