Re:ParaText: CSV parsing at 2.5 GB per second
by bariumbitmap on Wednesday June 08, 2016 @01:25PM
(#11863905)
One of the problems with this mechanism is data entry. It requires specialized software to insert an RS character instead of a newline ('\n') when going to the next row, for example.
After it has been saved in that format, it is a text file consisting of one very long line (unless some of the fields have newlines, but that doesn't represent the actual structure). This makes it inconvenient to browse with a text editor or pager. Sure, you could have a special mode to make them show up as newlines, but then you can't tell the difference between real newlines and newlines inserted for convenience.
Consequently, Unix text-processing tools like cut, sed, and awk use a line feed ('\n') as a record delimiter, unless there is a special reason not to (e.g. the -print0 flag on find).
After it has been saved in that format, it is a text file consisting of one very long line (unless some of the fields have newlines, but that doesn't represent the actual structure). This makes it inconvenient to browse with a text editor or pager. Sure, you could have a special mode to make them show up as newlines, but then you can't tell the difference between real newlines and newlines inserted for convenience.
Consequently, Unix text-processing tools like cut, sed, and awk use a line feed ('\n') as a record delimiter, unless there is a special reason not to (e.g. the -print0 flag on find).
Related discussion here:
https://news.ycombinator.com/item?id=7474600