News for hackers. Stuff that matters.
Re:Modernizing AWK, a 45-year old language, by adding CSV su... by mro_name on Thursday May 12, 2022 @06:47AM (#31352049)
I recently learned via https://news.ycombinator.com/item?id=31257248 that ASCII has the idea of records and fields ever since. It's just not used, but workaround CSV.

No improvement of CSV handling will ever improved on that.

Re:Modernizing AWK, a 45-year old language, by adding CSV su... by belter on Thursday May 12, 2022 @07:28AM (#31352401)
Discussed here before, not everybody was seeing only positives.

"ASCII Delimited Text – Not CSV or TAB delimited text": https://news.ycombinator.com/item?id=7474600

Re:Modernizing AWK, a 45-year old language, by adding CSV su... by Chris2048 on Thursday May 12, 2022 @07:37AM (#31352483)
A big problem is tooling - what they currently support, and the fact that excel will f*ck up anything.

I suppose the best way would be a suite of tools to edit them that are compatible with existing editors? TBH a great value-add for adoption would be versioning s.t. you can see (and revert) when other tools mess up your files.

Re:Modernizing AWK, a 45-year old language, by adding CSV su... by cb321 on Thursday May 12, 2022 @06:59AM (#31352170)
See the "Conventions for lossless conversion to TSV" on the current version of https://en.wikipedia.org/wiki/Tab-separated_values . There are really only 3 chars to escape - the escape char, and 2 delimiters - to make everything easy to deal with (even binary data in fields - what "lossless" means here).
Re:Modernizing AWK, a 45-year old language, by adding CSV su... by carapace on Thursday May 12, 2022 @11:05AM (#31355262)
I'm just gonna leave this here.

https://git.sr.ht/~sforman/sv/tree/master/item/sv.py

(Python code for ASCII-separated values. TL;DR: it's so stupid simple that I feel any objection to using this format has just got to be wrong. It's like "Hey, we don't have to keep stabbing ourselves in the face." ... "But how will we use our knives?" See? It's like that.)

Re:Modernizing AWK, a 45-year old language, by adding CSV su... by a1369209993 on Friday May 13, 2022 @12:53AM (#31363476)
> any objection to using this format

How do I edit the data file in a text editor?

More generally, how do I edit the entire data file, including field/record/etc seperators, in a editor that can display only characters that are valid content of data fields?

(Not that CSV (barring de-facto-nonstandard extensions like rfc4180's `"b CRLF bb"` and `"b""bb"`) is any good for this either, of course.)

Re:Modernizing AWK, a 45-year old language, by adding CSV su... by CRConrad on Friday May 20, 2022 @04:30AM (#31444771)
> > any objection to using this format

> How do I edit the data file in a text editor?

Use a non-sucky text editor.

For example, AFAICR NotePad++ displays those characters. Windows doesn't have any ultra-easy way to input them, but within NP++ you can copy-paste them.

Re:Modernizing AWK, a 45-year old language, by adding CSV su... by a1369209993 on Saturday May 21, 2022 @05:38PM (#31462127)
> For example, AFAICR NotePad++ displays those characters.

Great! Now go back to the other half of the requirement: how do I store all characters that NotePad++ displays in the content of a data field?

Re:Modernizing AWK, a 45-year old language, by adding CSV su... by CRConrad on Monday May 23, 2022 @02:47AM (#31476012)
These delimiters, you mean? You don't. They're delimiters, they never go in a field. Bam, problem solved. (Or rather, problem didn't exist in the first place.)
Re:Modernizing AWK, a 45-year old language, by adding CSV su... by mro_name on Saturday May 14, 2022 @05:57AM (#31376424)
indeed, if the text editor doesn't care, it's no help. Same for all other characters - be it tab, cr, lf, crlf, ä, and so on.

Choose your tools wisely, I'm not aware the situation.

Re:Modernizing AWK, a 45-year old language, by adding CSV su... by RhysU on Thursday May 12, 2022 @07:52AM (#31352616)
Could one get away with regular AWK using these combined with a streaming to/from CSV converter?
Re:Modernizing AWK, a 45-year old language, by adding CSV su... by mmcgaha on Thursday May 12, 2022 @10:57AM (#31355171)
Yes and I suspect that this is why it was never added to AWK. I,and I am sure most people, have an AWK filter to transform their csv or whatever format to a format that AWK can use with an appropriate FS and RS.

Their "but why" section should really go into more detail about why filters are not 100% if your data has the possibility of containing your preferred line/record separator.

But real world, I have never had a situation where a csv-to-awk filter (written in awk of course) did not work.

Re:Modernizing AWK, a 45-year old language, by adding CSV su... by cb321 on Thursday May 12, 2022 @08:15AM (#31352858)
Not just "regular awk", but "regular most things", like head, tail, grep, etc. The "things" just need to be 8-bit/NUL handling clean and the data could even have binary fields. I mean, if there is a library like Python/Go/etc. that you especially trust to write a streaming converter then that is a fine approach, but otherwise see both https://news.ycombinator.com/item?id=31352517 and https://news.ycombinator.com/item?id=31352704 (at least!)
Re:Modernizing AWK, a 45-year old language, by adding CSV su... by andylynch on Thursday May 12, 2022 @08:28AM (#31353019)
Closely related- I remain mystified as to why FIX Protocol chose to use control characters to separate fields, but used SOH rather than RS/US or the like.
Re:Modernizing AWK, a 45-year old language, by adding CSV su... by kevin_thibedeau on Thursday May 12, 2022 @12:46PM (#31356591)
Because of EBCDIC.