Re:Tad, a tabular data viewer
by elcritch on Tuesday May 30, 2017 @11:15PM
(#14450189)
Thanks for putting this out there!
There are some projects out there using memory mapped files to do fast CSV parsing. Could be a nice way to speed up the memory loading and scroll it in real time. Can't find the link to the library I saw it used in, but it might be an interesting venue to consider. Another library that does it seems to be astropy fast ascii IO module [1].
Re:Tad, a tabular data viewer
by camkego on Wednesday May 31, 2017 @05:27AM
(#14451258)
Try benchmarking OS read() calls vs. either sequential or random reads using memory mapping, whenever I do this OS read() calls end up being quite a bit faster.
There are some projects out there using memory mapped files to do fast CSV parsing. Could be a nice way to speed up the memory loading and scroll it in real time. Can't find the link to the library I saw it used in, but it might be an interesting venue to consider. Another library that does it seems to be astropy fast ascii IO module [1].
[1]: http://docs.astropy.org/en/stable/io/ascii/fast_ascii_io.htm...