Re:SQLite as an Application File Format (2014)
by shabbyrobe on Tuesday December 25, 2018 @01:46AM
(#18756304)
I've always been curious about the characters in ASCII for this, but I've never seen them used in the wild. Stuff like "Group Separator" (0x1D), "Record Separator" (0x1E) or "Unit Separator" (0x1F)
Is there a reason why nobody uses these? Did someone work out back in the 90s they were pure evil and we've just never used them since?
Re:SQLite as an Application File Format (2014)
by mikestew on Tuesday December 25, 2018 @09:22AM
(#18757511)
https://www.lammertbies.nl/comm/info/ascii-characters.html
Those codes were originally for just such a purpose, but as others point out there was a bootstrapping problem. At this point, if Excel doesn’t support it, it’s not going to gain traction.
Re:SQLite as an Application File Format (2014)
by coredog64 on Tuesday December 25, 2018 @11:24AM
(#18757920)
About 10 years ago I had to use those for a financial system integration. I was getting files that had been created on a mainframe, and whoever wrote it originally had the foresight to use those characters. Probably because they were based out of EMEA and understood that commas weren’t useful across national borders.
Re:SQLite as an Application File Format (2014)
by GnarfGnarf on Tuesday December 25, 2018 @08:13AM
(#18757273)
The codes between x01 and x1F were designed for telecommunications, for instance binary custom formats over RS-232 or synchronous protocols.
There is an excellent description in "C Programmer's Guide to Serial Communications" by Joe Campbell.
Re:SQLite as an Application File Format (2014)
by euos on Tuesday December 25, 2018 @02:49AM
(#18756465)
There’s no physical key for them. You can trivially write/edit CSV or TSV on any computer using any editor.
Re:SQLite as an Application File Format (2014)
by mirimir on Tuesday December 25, 2018 @03:00AM
(#18756491)
Indeed. I used to clean up data in UltraEdit. With the option for direct disk read/write set, there was virtually no file-size limit.
Re:SQLite as an Application File Format (2014)
by mirimir on Tuesday December 25, 2018 @01:51AM
(#18756318)
When I was working as a forensic data analyst, I felt pretty much the same. If you request CSV in discovery, there's no telling what you'll get. I mean, the data may come from custom COBOL, and then get reviewed by someone using Excel.
So yes, TSV. However, I've seen TSV with spurious tabs :(
Sometimes I ended up pushing for |-delimited data. Or even fixed-width format :)
When sharing tabular data in text format, he always preferred TSV because commas were everywhere in the material we were working with, but tab characters were really rare.