News for hackers. Stuff that matters.
Re:CleverCSV: A Drop-In Replacement for Python's CSV Module by donarb on Monday January 13, 2020 @05:20PM (#22039366)
Pandas uses the Python csv module under the hood. Most of the code in the read_csv method is used to get the data from the csv file and load it into data frames.
Re:CleverCSV: A Drop-In Replacement for Python's CSV Module by urschrei on Monday January 13, 2020 @06:52PM (#22040146)
Are you sure? Wes McKinney rewrote the Pandas CSV parser way back in late 2012 because the batteries-included version was slow and memory-hungry. Did Python upstream his version later on, or replace it with something faster (I'd be surprised if they did either, but I'd be curious to know)
Re:CleverCSV: A Drop-In Replacement for Python's CSV Module by evanpw on Monday January 13, 2020 @09:50PM (#22041352)
It's configurable. The default parser is written in C, but it's possible to use a parser based on the csv module if you needs things like separators specified by a regex.