News for hackers. Stuff that matters.
Re:Friends don't let friends export to CSV by josefx on Tuesday March 26, 2024 @02:08AM (#39824688)
Does it have nesting operators? I want to embedd ASCII within my ASCII fields. So I can have a table within my table.
Re:Friends don't let friends export to CSV by vaylian on Tuesday March 26, 2024 @02:27AM (#39824768)
It kind of does. See `man ascii`

* FS (0x1C) file separator

* GS (0x1D) group separator

* RS (0x1E) record separator

* US (0x1F) unit separator

I've never seen these in the wild though.

Re:Friends don't let friends export to CSV by alex_suzuki on Tuesday March 26, 2024 @02:36AM (#39824809)
They‘re used a lot in barcodes, e.g. for delimiting the different fields of a driving license.
Re:Friends don't let friends export to CSV by euroderf on Tuesday March 26, 2024 @04:17AM (#39825297)
That sounds like the premise for an utterly fascinating deep dive.
Re:Friends don't let friends export to CSV by alex_suzuki on Tuesday March 26, 2024 @12:23PM (#39829661)
It‘s quite the rabbit hole, I can assure you.
Re:Friends don't let friends export to CSV by adammarples on Tuesday March 26, 2024 @08:15AM (#39826886)
Not seeing them in the wild is good, it means they will work when you use them. The more they get used the more often you'll find they crop up in the text fields you're trying to separate and the whole csv escaping nightmare will continue.
Re:Friends don't let friends export to CSV by Repulsion9513 on Tuesday March 26, 2024 @05:31PM (#39833111)
If you've got delimited-text embedded inside your delimited-text, you've got a nightmare that escaping can't save you from.

(obligatory https://knowyourmeme.com/memes/xzibit-yo-dawg)

Re:Friends don't let friends export to CSV by dhagberg on Tuesday March 26, 2024 @01:02PM (#39830184)
They're definitely used in ACARS messages that go to every commercial airplane cockpit...
Re:Friends don't let friends export to CSV by lolive on Tuesday March 26, 2024 @02:19AM (#39824734)
A manager: I wish I could have a CSV inside my CSV.

Any sane person: NO!

Re:Friends don't let friends export to CSV by bruce511 on Tuesday March 26, 2024 @04:07AM (#39825241)
Well you might be wrong, but EDI in general and HL7 specifically allow 3 levels of "fields in fields in field".

As long as your parser copes, and as long as you have appropriate structures to import into, its no big deal.

Re:Friends don't let friends export to CSV by iainmerrick on Tuesday March 26, 2024 @04:52AM (#39825547)
So now when I'm exporting data I need to know what nesting level it's going to live at so I can generate the correct separators?

I really think that might be the worst idea I've heard for a while!

Re:Friends don't let friends export to CSV by bruce511 on Thursday March 28, 2024 @10:52AM (#39852329)
It's not toooo bad :) But it's very much a "thing" in the real word. It's called EDI, and it's been around for a long time.
Re:Friends don't let friends export to CSV by josefx on Tuesday March 26, 2024 @02:27AM (#39824771)
base64 encoded fields it is.
Re:Friends don't let friends export to CSV by lolive on Tuesday March 26, 2024 @02:36AM (#39824806)
That was the right answer. You are hired!
Re:Friends don't let friends export to CSV by wiredfool on Tuesday March 26, 2024 @04:42AM (#39825473)
Been there, done the Interop.
Re:Friends don't let friends export to CSV by icedchai on Tuesday March 26, 2024 @09:47AM (#39827770)
About 10 years ago, I worked at a place where we were embedding both XML and JSON in CSV fields.

Then there are always the people who can't generate a valid CSV due to escaping issues...

Nothing is ever simple.

Re:Friends don't let friends export to CSV by ykonstant on Tuesday March 26, 2024 @05:28AM (#39825773)
The POSIX spec defines portable and non-portable ASCII characters, prudently placing the separators in the non-portable set. In order to nest tables, base64 (or whatever portable encoding) code the table into the field. This works much better, easier and more error-free than any escaping strategy.

Regarding visibility in editors, if you are nesting tables I don't think you care too much about manual editing, but if you do, it is easy to setup vim/emacs/vscode to display the ASCII separators. I am being told even notepad++ can display those, there are no excuses.