Re:Ask HN: What do you use to open large CSV files?
by jklein11 on Thursday August 03, 2017 @10:29AM
(#14919856)
If you can do what ever you want to do by analyzing one line at a time I would recommend reading the file as a stream for processing.
If you can't do whatever you'd like one line at a time, read the file as a stream and write it to disk with some database(e.g sqlite, MySQL, Mongo). This will be slower but might be easier to play with the data.
If you can't do whatever you'd like one line at a time, read the file as a stream and write it to disk with some database(e.g sqlite, MySQL, Mongo). This will be slower but might be easier to play with the data.