Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha...
by StefanKarpinski on Sunday October 11, 2020 @07:49PM
(#24750124)
> there inherently is no optimal solution to "a problem with large performance-correctness tradeoffs"
You seem to be under the misapprehension that mainstream CSV libraries parse float values with reduced precision in order to go faster. They do not. A decimal floating-point string represents a precise mathematical value and the only acceptable value for a CSV parser to produce is that value correctly rounded to a Float64 (or some other type if requested). Any CSV parser that does anything else is simply incorrect.
> We saw elsewhere in the thread that Julia's float parsing code at least presently handles certain syntaxes incorrectly.
You may be referring to a bug mentioned elsewhere that fields like `-123.` with a leading minus sign and a trailing decimal point are misinterpreted as strings. That is a bug, plain and simple. It has nothing to do with float parsing speed/accuracy tradeoffs.
You seem to be under the misapprehension that mainstream CSV libraries parse float values with reduced precision in order to go faster. They do not. A decimal floating-point string represents a precise mathematical value and the only acceptable value for a CSV parser to produce is that value correctly rounded to a Float64 (or some other type if requested). Any CSV parser that does anything else is simply incorrect.
> We saw elsewhere in the thread that Julia's float parsing code at least presently handles certain syntaxes incorrectly.
You may be referring to a bug mentioned elsewhere that fields like `-123.` with a leading minus sign and a trailing decimal point are misinterpreted as strings. That is a bug, plain and simple. It has nothing to do with float parsing speed/accuracy tradeoffs.