Re:XSV: A fast CSV command-line toolkit written in Rust
by shawn on Sunday September 09, 2018 @08:27AM
(#17945580)
Agreed, though I'm mainly seeing how quickly I can reimplement everything xsv has to offer without sacrificing performance. I don't consider the challenge finished until, as you say, it handles all of the corner cases.
EDIT: I'm actually not sure what's going on with the startup time, since it's usually fast. I have quite a few windows open, which might be affecting results. (xsv count is suddenly taking ~70ms for simple data, so I think a reboot is in order.)
To clarify, I was mainly just excited to share some WIP. There's still a lot of work left to do to cross the finish line.
Re:XSV: A fast CSV command-line toolkit written in Rust
by burntsushi on Sunday September 09, 2018 @08:28AM
(#17945583)
Ah OK. Then to preserve our sanity, I'm going to bow out until you tell me I should go look. There are other things I want to accomplish today. :-)
Counting is interesting, because you don't have to implement unescaping to do it, but any robust csv parser will do it. So if you write two different versions of a csv parser, one for normal reading and one just for counting, then the one for counting can go faster and you'll avoid the need to amortize allocation. It's a nice trick though! However, I was using `xsv count` as a proxy for CSV parsing. So if you're just going to not do actual CSV parsing, then your solution is much less interesting. :-)
> I would definitely be curious to know why your program is taking 184ms though. That isn't startup time, is it?
> It's actually the time it takes to load in a C function to swap big-endian uint64 to little-endian.
Holy moly. Really? Do you know why it takes so long? That's a showstopper...