News for hackers. Stuff that matters.
Re:21 GB/s CSV Parsing Using SIMD on AMD 9950X by zeristor on Friday May 09, 2025 @11:45AM (#43938128)
Why not use Parquet?
Re:21 GB/s CSV Parsing Using SIMD on AMD 9950X by mcraiha on Friday May 09, 2025 @11:54AM (#43938218)
Excel does not output Parquet.
Re:21 GB/s CSV Parsing Using SIMD on AMD 9950X by speed_spread on Friday May 09, 2025 @01:07PM (#43939013)
True. But also Excel probably collapses into a black hole going straight to hell trying to handle 21GB of data.
Re:21 GB/s CSV Parsing Using SIMD on AMD 9950X by hermitcrab on Friday May 09, 2025 @01:53PM (#43939420)
Excel .xlsx files are limited to 1,048,576 rows and 16,384 columns.

Excel .xls files are limited to 65,536 rows and 256 columns.

Re:21 GB/s CSV Parsing Using SIMD on AMD 9950X by mihular on Friday May 09, 2025 @02:18PM (#43939654)
21GB/s, not 21GB ...
Re:21 GB/s CSV Parsing Using SIMD on AMD 9950X by anthk on Friday May 09, 2025 @02:53PM (#43939924)
mawk would handle a 21 GB csv (or maybe one true awk) fast enough.
Re:21 GB/s CSV Parsing Using SIMD on AMD 9950X by buyucu on Friday May 09, 2025 @02:13PM (#43939614)
Excel often outputs broken csv :)
Re:21 GB/s CSV Parsing Using SIMD on AMD 9950X by hinkley on Friday May 09, 2025 @06:30PM (#43941535)
I have been privileged in my career to never need to parse Excel output but occasionally feed it input. Especially before Grafana was a household name.

Putting something out so manager stops asking you 20 questions about the data is a double edged sword though. Those people can hallucinate more than a pre-Covid AI engine. Grafana is just weird enough that people would rather consume a chart than try to make one, then you have some control over the acid trip.

Re:21 GB/s CSV Parsing Using SIMD on AMD 9950X by constantcrying on Friday May 09, 2025 @02:49PM (#43939884)
Or HDF5 or any other format which is actually meant to store large amounts of floating point data.
Re:21 GB/s CSV Parsing Using SIMD on AMD 9950X by jsight on Friday May 09, 2025 @08:57PM (#43942382)
Because that would be too logical. :)

It is an interesting benchmark anyway.