News for hackers. Stuff that matters.
Re:New technique to easily partition CSV files for parallel ... by gazoduke on Wednesday February 25, 2026 @11:33AM (#47153816)
If I remember correctly CSV.jl also has something of the kind: https://csv.juliadata.org/stable/reading.html#CSV.Chunks

Used statistics are a bit different though.

Re:New technique to easily partition CSV files for parallel ... by topita on Wednesday February 25, 2026 @11:41AM (#47153968)
Isn't this more robust though? I feel like using lines to detect next rows is very flimsy. I usually deal with CSV containing full press articles, I am quite sure the CSV.Chunks method would fail without the correct hyperparameter. This method seems more, I dunno, "adaptative".