News for hackers. Stuff that matters.
Re:Time to retire the CSV? by Twisell on Wednesday August 18, 2021 @12:22PM (#28223473)
You got my vibe

Almost any current CSV parser, even the bad ones, tolerate a header line.

So it should be possible to define a compact and standardized syntax that is appended before the real header of the first cell (separator,encoding,decimal separator (often disregarded by most parsers but crucial outside USA),quote character,escape character,etc...). Following headers would just use special notation to inform on (data-type,length,comment).

Newest parsers would use theses clues, older ones would just append some manageable junk to headers.

Re:Time to retire the CSV? by Macha on Wednesday August 18, 2021 @12:54PM (#28223941)
So someone opens this CSV in Excel and there's garbage in A1?

Does this really count as compatible? You will get user bugs for this.

Re:Time to retire the CSV? by dragonwriter on Wednesday August 18, 2021 @01:01PM (#28224079)
> So someone opens this CSV in Excel and there's garbage in A1?

Yeah, that's why I chose the “thing that looks like a text file—including optionally CSV—but has additional metadata after the EOF mark” approach instead of stuffing additional metadata in the CSV; there's no way to guarantee that existing implementations will safely ignore any added metadata the main CSV body. (My mechanism has some risk in that there are probably CSV readers that treat the file as a binary byte stream and use the file size rather than a text stream that ends at EOF, but I expect its far fewer than will do the wrong thing with additional metadata before the first header.

Re:Time to retire the CSV? by Macha on Wednesday August 18, 2021 @01:16PM (#28224321)
If by EOF char you mean Ctrl-Z, Python's `csv` module is at least one case where it will read past the EOF char and you'll get rows of garbage data for any content in the file after that.