1.5 million rows you say? Better hope nobody opens it in Excel and overwrites it…

Hopefully the first column isn't ID either[1]

[1]: https://alunr.com/excel-csv-import-returns-an-sylk-file-form...

OpenOffice can’t print on Tuesdays https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161...

Software is hard...

Oh don’t. I spent several hours last week unfucking one of those.

Thank you, I'm considering adopting this as job title, like "senior unfucker".

Intermediate Fucker reporting for duty.

Mine is principal dung roller.

I’ve long described myself as half janitor, half firefighter.

I sometimes fill in my name as [object Object].

Data Plumber.

Some planning, some building out new stuff (usually clean work), some repairing active geysers of partially processed data that's getting all over the place fouling up the works.

Do tell! If you’ve recovered ;)

I have. All I'm saying is this plus someone who left 10 years ago who can't and shouldn't have written a CSV parser using regular expressions. Input row:

  A,B,Alice "Mallory" Bob,123

I just had one today that had the last char of a field as '\', which escaped the closing quote and then munged the column count from there.

Isn't that your parser's problem? I don't think true CSV has any special characters other than comma, quote, and CRLF.

I thought it strange too. I saw what the issue was, and just "fixed" it be correcting the data in the CSV. For the lulz, I guess I could have played with the parser's options on deciding what is needed to be escaped. However, the data would have still been incorrect as the '\' is definitely not part of the desired content, so ultimately it was better to correct the input. i would kind of rather the import die than having the potential foot gun of '\' in a field for later sabotage.

How do I name my child CRLF?

earn Musk money, you can name your child WTF you want

i think you failed to parse the technical joke

no, i totally got the line ending joke. i'm not an ID10T

Unless it's a CSV file exported from a Nordic locale Excel, in which case your CSV exports will use semi-colon as column separators and commas as decimal points. And yes the filename will still end with ".csv"

So the following Excel export I just did will parse perfect fine with your CSV parser but give you completely the wrong thing:

  1;2,3;3,3
  2;7,3;9,3
  3;4,5;7,5

That has nothing to do with the Nordics, but with the decimal separator. In locales that use a comma as the decimal separator (i.e., most European locales), Excel uses a semicolon as CSV separator.

* * *

I think it's valid to argue that you shouldn't be able to put some of commas, quotes, and newlines inside fields at all. And comma versus semicolon.

But that doesn't extend to using backslash escapes in something that's legitimately trying to be CSV. That's someone getting confused and implementing a mix of data formats, or trying to be clever and making an extended CSV format.

It’s valid to argue that, but that means you can’t use CSV for many real-world data sets.

That, in turn, means you almost cannot use CSV in any robust solution. Even if, today, your input doesn’t have commas, quotes or newlines, can you guarantee it won’t tomorrow, next year, etc?

Yes? What do I not know that makes this question harder than it seems to me?

> Even if, today, your input doesn’t have commas, quotes or newlines, can you guarantee it won’t tomorrow, next year, etc?

But... but those are the ones I listed as real special characters, unlike backslash. I don't understand the question.

Every field is quoted. Every field is quote-escaped. That's all you need.

There are simpler and better approaches. Fortunately for us all, IP datagrams are not JSON-encoded, for example.

If I’m not mistaken RFC 4180 says that quotes should be escaped by prepending them with another quote, so “” and not \” (these are not double quotes but my phone won’t let me type normal quotes), but yeah I guess it is a rather perverse value to put in a csv.

Note that virtually no one implements that RFC. CSV is merely a style of file, not a true file format.

That's how Excel does it though