Re:Why I sometimes like to write my own number crunching code
by im3w1l on Saturday June 26, 2021 @10:58AM
(#27642411)
So uhm why did the cosine similarity give values bigger than one in the library and how can we be confident that this code does not have that issue?
[ Parent
| on HN ]
Re:Why I sometimes like to write my own number crunching code
by username90 on Saturday June 26, 2021 @10:31PM
(#27647988)
The answer is 10 / 14, which expands to his answer if you write out the decimals. The library just truncated it in its output. Doing your own cosine similarity is really hard to mess up.
Re:Why I sometimes like to write my own number crunching code
by im3w1l on Saturday June 26, 2021 @10:51PM
(#27648093)
I guess I'm concerned with rounding errors in the intermediate results leading to a similarity of 1.00000001 or something. When you say 10/14, I don't know what you are referring to btw. I couldn't easily find the original bug or library.
Re:Why I sometimes like to write my own number crunching code
by username90 on Saturday June 26, 2021 @11:03PM
(#27648161)
> When you say 10/14, I don't know what you are referring to btw.
I did the calculation manually, the answer is exactly 10 divided by 14. Basically you just take ( 1 * 3 + 2 * 2 + 3 * 1 ) / (3 * 3 + 2 * 2 + 1 * 1), which is 10 / 14. Computing cosine similarities is really simple as I said.
Edit: The funny thing is that his answer is actually a bit too low since he truncates instead of rounds at the end. 10 / 14 just continues as 142857 and repeats after that. So if the library is smaller than his then the library is wrong and not him. Anyway, blindly trusting the results of a library is just dumb, they make mistakes so often you are right and they are wrong.
Re:Why I sometimes like to write my own number crunching code
by dragandj on Sunday June 27, 2021 @05:00AM
(#27649852)
The matrix library only truncates the printout of matrix entries. The output is, of course, full 32bit or 64bit floating point number.
Re:Why I sometimes like to write my own number crunching code
by tectonicfury on Saturday June 26, 2021 @01:14PM
(#27643732)
good question