News for hackers. Stuff that matters.
Re:Show HN: uDSV.js – A faster CSV parser in 5KB (min) by liuliu on Wednesday September 06, 2023 @09:51PM (#37413760)
With multi-thread and some tricks, csv parsing should be able to reach SSD speed (~2GiB/s read): https://liuliu.me/eyes/loading-csv-file-at-the-speed-limit-o...

At that point, I think the speed is limited by boxing JS objects.

Re:Show HN: uDSV.js – A faster CSV parser in 5KB (min) by leeoniya on Wednesday September 06, 2023 @09:56PM (#37413792)
nice, is this something you've tried in WASM or WebGL/WebGPU?

at some point it makes sense to simply recommend using WASM SQLite (or DuckDB) to load the data for OLAP stuff. im not gonna encourage anyone to use JS on multi-GB files. even a few hundred MB becomes questionable ;)

Re:Show HN: uDSV.js – A faster CSV parser in 5KB (min) by texodus on Wednesday September 06, 2023 @10:20PM (#37413956)
The CSV parser for [Perspective](https://github.com/finos/perspective) uses the [Apache Arrow](https://github.com/apache/arrow) C++ CSV parser compiled to WASM. It's not currently multi-threaded but this is possible as well to my understanding.
Re:Show HN: uDSV.js – A faster CSV parser in 5KB (min) by leeoniya on Wednesday September 06, 2023 @10:24PM (#37413987)
what i learned from benchmarking Dekkai, is that simply "being in WASM" is a poor indicator of actual performance if you ever need to pull the values across that boundary into JS...especially strings.

i'll take a look and see if i can incorporate Perspective's parser into the bench, thanks!

Re:Show HN: uDSV.js – A faster CSV parser in 5KB (min) by tomcam on Wednesday September 06, 2023 @11:58PM (#37414599)
What does it mean to box an object?
Re:Show HN: uDSV.js – A faster CSV parser in 5KB (min) by davefol on Thursday September 07, 2023 @12:30AM (#37414781)
I believe allocating memory for objects on the heap. https://stackoverflow.com/a/80113/10654749
Re:Show HN: uDSV.js – A faster CSV parser in 5KB (min) by tomcam on Thursday September 07, 2023 @02:37AM (#37415483)
I think it’s more specific, something along the lines of turning a reference (pointer-allocated) object to a value-based object? But I’m woefully ignorant on this.
Re:Show HN: uDSV.js – A faster CSV parser in 5KB (min) by liuliu on Monday September 11, 2023 @01:50PM (#37470903)
Oh, by "boxing object", I meant to create the in-memory object that can be used by JavaScript runtime, preferably native objects. For example, in JSC case, something like https://developer.apple.com/documentation/javascriptcore/jsv....
Re:Show HN: uDSV.js – A faster CSV parser in 5KB (min) by tomcam on Saturday September 16, 2023 @08:03AM (#37534071)
Ah, thanks