I was not aware of the scorn for the style, I just knew I enjoy reading them.
As opposed to your μTSV may I suggest DSV. The Art of Unix Programming makes a strong case for how it's superior to CSV [1]
1. http://www.catb.org/esr/writings/taoup/html/ch05s02.html#id2...
Processing DSV data (Delimiter-Separated Values) with Python:
https://jugad2.blogspot.in/2016/11/processing-dsv-data-delim...
The example program in the post lets you use either a "-c delimiter_char" or a "-n delimiter_ASCII_code" to specify the delimiter for your DSV data being processed by it.
and
[xtopdf] Publish DSV data (Delimiter-Separated Values) to PDF:
https://jugad2.blogspot.in/2016/12/xtopdf-publish-dsv-data-d...
Bah. How am I wrong when I haven't even claimed DSV to be the best? Just quoting that section in ESR's book does not imply I endorse or agree with it. Logic ...
I don't have objective proof that it's the best. It's a mix of personal preference and empirical "evidence" based on past success with both EDN and its granddaddy s-expressions for data exchange and serialization.
As for why I personally like EDN so much:
1. It offers the malleability, flexibility, and extensibility of XML, while being
2. even more concise than JSON,
3. straightforward to parse, and
4. precisely defined in a rigorous and language-independent specification.
Traits 1-3 it shares with the traditional s-expressions of the Lisp family, but in contrast to s-expressions it's specifically designed as a data exchange format rather than as a lexical/syntactic format for programming languages. The reason this matters is because the traditional Lisp reader is itself extensible, and extensions can't be guaranteed to carry over from dialect to dialect or even from implementation to implementation. Many Lisp readers go so far as to even allow the execution of arbitrary code at read time, which is desirable for source code as it enables metaprogramming, but it's not so desirable when you're parsing data from an arbitrary source, due to security concerns.
While EDN does have its roots in Clojure's reader, EDN is not exactly Clojure's external representation. Rather, it's a subset of Clojure's surface syntax in much the same way that JSON is a subset of JavaScript's. Like JSON, EDN works great outside of its "native language" (in fact, I've never used EDN from Clojure itself; I've only used it from Lua, Scheme, Python, C, C++, C#, Java, and JavaScript (not necessarily in that order)).
I understand, and agree it can be difficult. Thanks for clearing that up, and no worries at all. Interesting info, BTW, about EDN. Will reply again in more detail later today with any comments/questions.
I was an early adopter of EDN, and wound up rolling my own parser in Python. EDN is simple enough that you can realistically roll a parser for it, complete with tests, in one or two working days. I find this to be a huge advantage—EDN doesn't really have any corner cases, so implementation is very straightforward.
And if you're using tabs as field delimiters then why do you need quotation marks in the mix?
By this standard, CSV is a terrible format.
#!CSV: escape=:\\', quote='"', quotealg='c-like', separator='\t', eof='\n\r', comment='#', encoding='utf-8'
foo bar baz
EDIT: The right solution is the simple one. Follow RFC4180 exactly. Reject everything that doesn't match it.
So, yes, follow RFC4180 strictly in your own code, but be able to read CSV files produced by others too. Single line of metainformation can greatly reduce confusion and is easy to throw out, when it is unnecessary.
I think I've said this in another thread, but I think rejecting malformed documents even though you could "figure out what they meant" is the equivalent of not telling people when they have food on their face.
I think the right pattern for something like this is to complain loudly when you are given a malformed document, then say, "If you really want me to parse this malformed document, use --force and I'll try to make some guesses, but really you should just ask for a well-formed document."
That is unrelated to well-formed documents in incompatible formats, though.
With that said, if you are interested in using CSV for encoding and decoding in your own environment, I suggest a new file standard called μTSV.
1. All μTSV files are UTF-8.
2. All values are delimited by "\t", and all lines must end with "\n" (including the last one).
3. If you want to use a "\t" or "\n" character in your value, tough luck---use JSON.