Never write your own parser, especially as just string.split(), and when possible don't use C(omma)SV formats, but C(haracter)SV, aka DSV: https://en.wikipedia.org/wiki/Delimiter-separated_values
There are non-printable, non-typable characters specifically defined as separators (ASCII 28-31) with UTF equivalents.
Re:Time to retire the CSV?
by 7thaccount on Wednesday August 18, 2021 @11:17PM
(#28229965)
As printed above, I'm not writing software where that would be a problem. In general if I was writing a commercial application or something in production where I don't control all the inputs I would agree 100%, but I'm lucky to not have those problems. I could use a bunch of libraries and additional code to try to catch non-existent errors I don't have, or fix my actual problems and move on.
I appreciate the perspective though for sure. I'm guilty of the same thing on HN, where I assume people have the same uses as me when they're writing code that has to be extremely robust or blazingly fast.
Re:Time to retire the CSV?
by 7thaccount on Wednesday August 18, 2021 @11:10PM
(#28229927)
You've assumed an awful lot about my use cases. The data I deal with in .CSV form is always pre-processed and doesn't have any of the minefield occurrences you've mentioned. There can't be a newline or anything like that in an input. In my decade of using .CSV files daily, I've only had one tertiary system where that is a problem.
Also, when doing interactive work, it's a bit different than writing production IT software.
So you're not actually parsing CSV and your comment was off-topic. Thank you for the clarification.
Re:Time to retire the CSV?
by 7thaccount on Thursday August 19, 2021 @05:14PM
(#28239455)
Define parsing. I'm still going through GB of data in thousands of files and building complex reports, and data structures for scientific analysis. Just because I don't need thousands of lines of code to navigate edge cases doesn't mean I'm not parsing .CSV files.
...until there is a newline inside a field.
The moronic quoting mechanism of CSV is one half of the problem; people like you, who try to parse it by "just splitting strings" is the other half. The third half is that it's locale dependent and after 30+ years, people still don't use Unicode.