Data scientists need to learn about significant digits

Re:Data scientists need to learn about significant digits by yodon on Monday December 25, 2023 @06:50PM (#38767465)

It's not uncommon to have 10m accuracy with consumer gear, which is 4 digits to the right of the decimal in degrees, and differential GPS can do around 3 orders of magnitude better, or around 7 digits. That's hitting the limits of float precision. By the time you start doing calculations and error propagation on the calculations, doubles are absolutely reasonable choices.

Are all location estimates that accurate? No. Should the precision of your location estimate be limited by your storage format? Of course not. That's why doubles are frequently used.

[ Parent | on HN ]

Re:Data scientists need to learn about significant digits by dmd on Monday December 25, 2023 @07:02PM (#38767560)

The point isn’t how you store them. The point is how you display them.

[ Parent | on HN ]

Re:Data scientists need to learn about significant digits by defrost on Monday December 25, 2023 @07:30PM (#38767764)

Relative to the quality of the instrumentation chain.

I've heard that some on this site do spend time measuring plate drift and correcting for atmospheric path distortion on long soak GPS base stations (that spend months recording GPS data for a fixed position to smooth out satellite and path error).

Your point stands though, there's no sense displaying drama digits that lack meaning in context.

[ Parent | on HN ]