News for hackers. Stuff that matters.

A comment on CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R. Parent comment

Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by karbarcca on Sunday October 11, 2020 @12:47PM (#24747087)
Take a deep look at Julia! Adding multithreaded parsing capabilities to CSV.jl was really a joy; basically just chunking up the file and spawning threaded tasks to process each chunk with the existing parsing code.

My favorite favorite thing about developing in Julia is the ability to write "high-level" type code and usually get decent performance, BUT THEN have the ability to fine tune for that extra boost of performance. The code inspection tools (see lowered, typed, LLVM, or native IR code levels) and super nice Task-based multithreading really make performance tuning a delight.

[ Parent | on HN ]
Re:CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster tha... by willvarfar on Sunday October 11, 2020 @12:54PM (#24747158)
(Some of the comments in https://news.ycombinator.com/item?id=24736559 talk about new lines inside fields, making multi-threading difficult)

(My quick search to find that recent story on HN made me find this too https://news.ycombinator.com/item?id=24740527)

[ Parent | on HN ]