News for hackers. Stuff that matters.
Re:ParaText: CSV parsing at 2.5 GB per second by arjie on Wednesday June 08, 2016 @11:44AM (#11862985)
I used to religiously believe in this, but in practice it isn't useful. The whole point is to be roughly human-readable and using non-printing characters defeats that. You can't even easily enter these things via the command line.

If we're abandoning human-readability, why even bother with ASCII? Just use a binary format. Has anyone actually used ASCII unit and record separator delimiters successfully? I'd be curious about what advantages they had over a binary format, even just a protobuf or Thrift serialized form. If we want to preserve schemalessness, there's stuff like Sereal.

Re:ParaText: CSV parsing at 2.5 GB per second by nostrademons on Wednesday June 08, 2016 @05:24PM (#11865786)
> Has anyone actually used ASCII unit and record separator delimiters successfully?

At my first job, the (proprietary) database server we used had a communication protocol that used the ASCII STX/ETX/EOT/ACK/ETB/GS/RS/US characters. When I looked up those ASCII codes I was like "Oh, that's clever", but in practice, it was just another proprietary binary format, and had all the problems of proprietary binary formats. The control codes still needed to be escaped in source code; they still weren't visible in log files or when viewing console output. They were basically just bytes we had to send to implement the protocol.

Re:ParaText: CSV parsing at 2.5 GB per second by zeveb on Wednesday June 08, 2016 @12:51PM (#11863613)
> The whole point is to be roughly human-readable and using non-printing characters defeats that.

You're assuming that separator characters are not human-readable. If ASCII had been used as originally intended, they'd be just as readable as line breaks. Err, carriage returns.

Computing depresses me some days …

Re:ParaText: CSV parsing at 2.5 GB per second by nostrademons on Wednesday June 08, 2016 @05:16PM (#11865735)
Most people build systems to work in the world we have, not the world we wish we had.

There are all sorts of examples like this. UNIX was initially designed to work with pipes of line-oriented streams, so why did we get scripting languages where every UNIX command is reinvented as a function call or RPC frameworks where the pipe is replaced by a binary message? PHP was initially a templating language, so why did we get Smarty, Wordpress, and PEAR templates? The web was supposed to come with full support for editing & creating pages via WYSIWYG editor and have a built in mechanism (hyperlinks) for associating pages with other people, so why did we need Facebook to introduce the idea of "sharing content with other people".

In each case, there were real, pragmatic reasons that people invented new systems instead of doing what they were "supposed" to do. Line-oriented files are clumsy for representing hierarchical data or conditionals. PHP is too hard to use for most end-users, despite being built for pragmatic "just toss a webpage up" use. The editing features in the web disappeared early on, with Netscape, and they needed the critical mass of college students that Facebook provided before people felt they had an audience for anything they did.

The moral for system designers is that you can't just throw a feature out there and say "Use this." You have to adapt it to how people actually do use it, even if that usage seems brain-dead to you.