Re:Future instruction set: AVX-512
by sixbrx on Wednesday October 09, 2013 @09:46PM
(#6525196)
To amplify this a bit, in binary the relative and absolute errors are within a factor of two, but by a factor of up to 10 for decimal. IBM actually used to use base 16 on some hardware but the rel/abs error disparity of factor 16 hurt and it was eventually abandoned.
[ Parent
| on HN ]
There are no comments on this comment yet.