News for hackers. Stuff that matters.
Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by RobinL on Monday October 12, 2020 @03:47AM (#24752298)
Apache Arrow reads csvs into memory in Arrow format, not pandas format. They are independent libraries that do different things. In Arrow, it is possible to read a csv in batches, obviating memory problems. See 'incremental reading' - http://arrow.apache.org/docs/python/csv.html

There are no comments on this comment yet.