News for hackers. Stuff that matters.

A comment on Working with CSV files on shell/terminal. Parent comment

Re:Working with CSV files on shell/terminal by wenc on Wednesday June 28, 2023 @01:49AM (#36502770)
My understanding is that the CSV is read in chunks, and then each chunk is parsed in a different thread.

This is the description in the experimental PR (parallel CSV has since merged into the main branch)

https://github.com/duckdb/duckdb/pull/5194

[ Parent | on HN ]
Re:Working with CSV files on shell/terminal by bombolo on Wednesday June 28, 2023 @07:07AM (#36504806)
Sounds like a good way to fail to parse all the possible files properly.
[ Parent | on HN ]
Re:Working with CSV files on shell/terminal by wenc on Wednesday June 28, 2023 @10:36AM (#36507096)
How so?
[ Parent | on HN ]
Re:Working with CSV files on shell/terminal by bombolo on Wednesday June 28, 2023 @04:43PM (#36512320)
Because you read a \n or a comma but you don't know if there was an open quote before the block you're currently reading.
[ Parent | on HN ]