CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by klhugo on Sunday October 11, 2020 @12:40PM (#24747024)

Here is the thing made me a happier person this year: I realized when CSVs take too long to load, I should not be using them in the first place!

I love SQLite now <3

[ Parent | on HN ]

Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by karbarcca on Sunday October 11, 2020 @12:43PM (#24747051)

It's always a transcendental experience to move data from CSV into any more structured/flexible format!

[ Parent | on HN ]

Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by klhugo on Sunday October 11, 2020 @12:53PM (#24747150)

And fast! I got easily 3 orders of magnitude speedup. I used to load CSVs with pandas and do typical filtering operations... oh boy, some of the stuff would take a hole morning, not to mention the memory usage. I do all that in minutes now.

[ Parent | on HN ]

Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by benhurmarcel on Monday October 12, 2020 @09:26AM (#24754113)

Sorry if that sounds stupid, but then when you get a csv file you write it to an SQLite file before doing computations?

[ Parent | on HN ]