Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha...
by RobinL on Monday October 12, 2020 @03:47AM
(#24752298)
Apache Arrow reads csvs into memory in Arrow format, not pandas format. They are independent libraries that do different things. In Arrow, it is possible to read a csv in batches, obviating memory problems. See 'incremental reading' - http://arrow.apache.org/docs/python/csv.html
[ Parent
| on HN ]
There are no comments on this comment yet.