ParaText: CSV parsing at 2.5 GB per second

Re:ParaText: CSV parsing at 2.5 GB per second by rspeer on Wednesday June 08, 2016 @12:28AM (#11860023)

Which means an effective thing to do when you control the data pipeline is to ban tabs and line-breaks from your values, and then use `cut` on tab-separated files.

[ Parent | on HN ]

Re:ParaText: CSV parsing at 2.5 GB per second by ygra on Wednesday June 08, 2016 @02:24AM (#11860382)

Sure, if you can redefine your data format, you can take shortcuts. But otherwise comparing cut to a CSV parser is just a different way of saying »if it doesn't have to be correct, I can make it as fast as I want«.

[ Parent | on HN ]

Re:ParaText: CSV parsing at 2.5 GB per second by Twirrim on Wednesday June 08, 2016 @01:04AM (#11860144)

Or just not use CSV as a data transfer format :D

[ Parent | on HN ]

Re:ParaText: CSV parsing at 2.5 GB per second by rspeer on Wednesday June 08, 2016 @11:52AM (#11863060)

I think you're missing the point. Unix tools like `cut` are really effective ways to deal with plain text data, who cares if it's CSV.

[ Parent | on HN ]