News for hackers. Stuff that matters.
Re:What Every Computer Scientist Should Know About Floating-... by ChrisLomont on Tuesday June 30, 2020 @03:43PM (#23693977)
>Almost all numerical problems software developers deal with can be represented exactly using rationals. Only a small fraction of all problems involve transcendentals.

There's more classes of real numbers than rationals and transcendentals. Hint: your example of sqrt(2) is neither.

As to doing things exactly with rationals, anytime you take a sin, cos, exp, log of any non-zero rational number the result is not rational. Any time you take a root of a non-perfect power rational you don't get a rational number (you get an algebraic number, which is neither rational nor transcendental).

So all you can do is simple arithmetic: + * / - (actually, that's from the definition of the field Q of rational numbers).

And you still run into the problem that rational number arithmetic gets arbitrarily slow and runs out of memory. Each multiply on average results in requiring the sum of the storage sizes of the multiplicands, making more than about 40 multiplications infeasible before you're out of RAM.

So your view of "almost all numerical problems software developers deal with" must be limited to only those using simple arithmetic and no more than around 40 operations. That's amazingly constraining.

What problems do you call numerical problems that fit your constraints?

Things that cannot be done exactly with rational numbers includes pretty much all of gaming (3D or otherwise), deep learning (probably every model has an exp in it), scientific computing, audio and video processing (cos and sin all over the place), and on and on.

>Using rationals does not mean that you can't round results.

Wait, you just claimed "...can be represented exactly using rationals. Now you want to round, throwing out the exactness? Which is it?

And congrats - you just re-invented floating-point math which is simply approximating values using rational numbers in a clever manner to handle larger ranges than fixed point. But they're still always rational approximations to values.

>I wrote that in mainstream languages should be able to evaluate (2^(1/2))^2 == 2. How do you go from there to requiring them to be able to evaluate infinite sums?!

Ah, so you mean languages should be able to simply solve this one specific problem, not all relatively simple symbolic problems? That seems like an even bigger mess than using floating point, which is well-defined and covers a large class of problems.

Does your mythical language know (A^(1/B))^B==A for all rationals A and B also representable in your language? Or are you limited to the value A=B=2 only?

(Hint: if you claim it should hold, you're gonna hit problems again :)

Re:What Every Computer Scientist Should Know About Floating-... by bjourne on Tuesday June 30, 2020 @05:46PM (#23695156)
> As to doing things exactly with rationals, anytime you take a sin, cos, exp, log of any non-zero rational number the result is not rational.

I fail to see your point. I never claimed that you can do real arithmetic with rationals.

> So all you can do is simple arithmetic: + * / - (actually, that's from the definition of the field Q of rational numbers).

Same as with floats. Except with floats you don't even get division.

> And you still run into the problem that rational number arithmetic gets arbitrarily slow and runs out of memory. Each multiply on average results in requiring the sum of the storage sizes of the multiplicands, making more than about 40 multiplications infeasible before you're out of RAM.

What?

    >>> from fractions import Fraction
    >>> Fraction(7, 8)**40
    Fraction(6366805760909027985741435139224001, 1329227995784915872903807060280344576)
In other words, no, you don't run out of memory after 40 multiplications...

> Things that cannot be done exactly with rational numbers includes pretty much all of gaming (3D or otherwise), deep learning (probably every model has an exp in it), scientific computing, audio and video processing (cos and sin all over the place), and on and on.

You sure can do all of those things with rational arithmetic. What you can't do is do them exactly but I never claimed you could.

> > Using rationals does not mean that you can't round results.

> Wait, you just claimed "...can be represented exactly using rationals. Now you want to round, throwing out the exactness? Which is it?

I wrote "ALMOST ALL numerical problems software developers DEAL WITH can be represented exactly using rationals." The key word in that sentence is "ALMOST."

> And congrats - you just re-invented floating-point math which is simply approximating values using rational numbers in a clever manner to handle larger ranges than fixed point. But they're still always rational approximations to values.

Yes, but I have also solved the freakishly annoying issues detailed in the article. Which one do you prefer:

    >>> print(sum(0.1 for _ in range(10)))
    0.9999999999999999
or

    >>> print(sum(Fraction(1, 10) for _ in range(10)))
    1
? I know which one users prefer.

> >I wrote that in mainstream languages should be able to evaluate (2^(1/2))^2 == 2. How do you go from there to requiring them to be able to evaluate infinite sums?!

> Ah, so you mean languages should be able to simply solve this one specific problem, not all relatively simple symbolic problems? That seems like an even bigger mess than using floating point, which is well-defined and covers a large class of problems.

I don't think Mathematica or the Python symbolic math packages I've worked with are particularily messy.

> Does your mythical language know (A^(1/B))^B==A for all rationals A and B also representable in your language? Or are you limited to the value A=B=2 only?

Yes, assuming A and B are positive.

Re:What Every Computer Scientist Should Know About Floating-... by ChrisLomont on Tuesday June 30, 2020 @11:12PM (#23697369)
> What? ... > In other words, no, you don't run out of memory after 40 multiplications...

So you think one case proves there is no set of 40 multiplications that overflows?

Since we're using a small starting value and only one operation per iteration, let's try 50 mults:

    x = frac(7,8)
    for i = 1 to 50
        x = x * x
    print(x) # as exact decimal fraction
Tell me how much ram and time this took you to evaluate with exact rational values. I'll wait :)

(Hint: you'll need approx 500 terabytes of ram to store these integers. I guess I didn't really need all 50 iterations! )

If you want only 40 mults, then using different starting values and adding a few additions will cause the same overflow. In fact, all sorts of common algorithms will overflow in very few operations using rationals. This is why numerical algorithms textbooks don't even cover it as a possibility - it's a terrible idea thrown out decades ago for precisely these reasons.

Your exact rational number idea has replaced predictable behavior on reasonable inputs with highly finicky unstable behavior, and a programmer will have no idea how to ensure things don't go off the rails for common uses.

In each reply you make fundamental math errors, provably incorrect claims, and don't stop to learn. I already gave a proof that such simple things will overflow - you ignored it, then tried to prove the converse with one toy example. It's not worth showing you more errors when you ignore evidence and continue disproven claims.

We're done. You're too stubborn to absorb relevant material.

Re:What Every Computer Scientist Should Know About Floating-... by bjourne on Wednesday July 01, 2020 @06:58AM (#23700101)
> So you think one case proves there is no set of 40 multiplications that overflows?

> Since we're using a small starting value and only one operation per iteration, let's try 50 mults:

That is repeated exponentiation, not repeated multiplication and you are grasping for straws. Try the same code using an integer or a float greater than 1. Since the same thing will happen as when using rationals (you get an error) are you going to tell me that those number types are useless too?

> In each reply you make fundamental math errors, provably incorrect claims, and don't stop to learn.

Now I realize that you are most likely trolling me. Fine. Have a nice day! Bye!