News for hackers. Stuff that matters.
Re:TSV Utilities: Command line tools for large, tabular data... by dev_dull on Saturday August 31, 2019 @08:59PM (#20849844)
We’ve discussed this at length and I’m squarely in the DO NOT camp. The drawbacks of using non-readable meta characters exceeds the benefits:

1. TSV can be read and imported by almost everything.

2. People can add and adjust TSV files from any editor.

3. What’s the way to insert meta characters again In VIM? And now nano? Argh I’ll just try and copy and paste it. Ugh that doesn’t work.

Just use CSV/TSV folks. Anything more complicated and reach for a better serialization format (json, yaml) and not a better delimiter.

Re:TSV Utilities: Command line tools for large, tabular data... by cbsmith on Saturday August 31, 2019 @09:04PM (#20849861)
2. More accurately, people can screw up TSV files from any editor. Have you not seen embedded spaces used instead of tabs and totally mess everyone up?
Re:TSV Utilities: Command line tools for large, tabular data... by cbsmith on Saturday August 31, 2019 @09:01PM (#20849850)
Like a lot of things, #1 is only partly true, particularly if you have embedded tabs.

The number of times I've had to deal with parsing errors because of embedded carriage returns, commas, tabs, etc., sometimes costing millions of dollars is just... upsetting.

It's like JSON... we say everything can parse it, but really what we've got is some approximation that will come back to haunt us... and once you've done all the work to make sure everything is precise and correct, you'd really have saved time if you'd just used the tools we already had in the first place.

Re:TSV Utilities: Command line tools for large, tabular data... by mruts on Saturday August 31, 2019 @09:27PM (#20849963)
Dealing with tabs and carriage returns has sometimes cost you millions of dollars? If so, that sounds like a great story and I would love to hear it!
Re:TSV Utilities: Command line tools for large, tabular data... by jlgaddis on Saturday August 31, 2019 @09:58PM (#20850078)
Especially if (as it sounds) it's happened more than once!
Re:TSV Utilities: Command line tools for large, tabular data... by Annatar on Sunday September 01, 2019 @04:56AM (#20851242)
Both JSON and YAML are very difficult to construct parsers for.

To make matters worse, both formats are full of pitfalls:

http://seriot.ch/parsing_json.php https://arp242.net/yaml-config.html

using either of these, in my opinion, is extremely misguided.

Re:TSV Utilities: Command line tools for large, tabular data... by chucksmash on Thursday September 05, 2019 @08:33AM (#20885968)
Can't find a source for it now, but one of the design goals of JSON iirc was that it should be easy to parse.

You should have a go at writing a JSON parser to convince yourself it isn't hard. Can manually lex it in less than 150 lines of Python.

Re:TSV Utilities: Command line tools for large, tabular data... by dev_dull on Sunday September 01, 2019 @10:48AM (#20852556)
The flip side to that is that json and yaml parsers exist in every language, and would be more than capable of replacing any logic you’d find in a CSV.
Re:TSV Utilities: Command line tools for large, tabular data... by Annatar on Monday September 02, 2019 @02:35AM (#20857174)
CSV's are a data structure. They do not contain any logic.
Re:TSV Utilities: Command line tools for large, tabular data... by tinus_hn on Tuesday September 03, 2019 @06:26PM (#20871886)
Just use these formats if you want to be stuck in the last century when internationalization was that odd thing you could easily afford to ignore. Otherwise, use a real format.
Re:TSV Utilities: Command line tools for large, tabular data... by cbsmith on Saturday August 31, 2019 @09:05PM (#20849864)
3. If you don't know how to use your editor, maybe you should learn. If your editor can't insert even all the characters in basic ASCII, it's not an editor.
Re:TSV Utilities: Command line tools for large, tabular data... by dev_dull on Saturday August 31, 2019 @09:41PM (#20850020)
Go ahead and give every engineer who comes across your meta-delimited file a little rundown on how it works. "Can you show me how to import it into google sheets?" "I need to email it to someone, can you show me how to change it?" "My IDE says I need a plugin to read it? Do you know anything about that? Can you help me set it up?"

They'll all agree how clever and useful the meta characters are of course, but only after you've given them your time in learning about it. No thanks no thanks no thanks. For me I'd rather deal with a little bit of serialization headache then a support headache.

Re:TSV Utilities: Command line tools for large, tabular data... by cbsmith on Saturday August 31, 2019 @11:56PM (#20850417)
...and this is why we can't have nice things.