Re:Command-line Tools can be 235x Faster than your Hadoop Cl...
by NavinF on Wednesday March 09, 2022 @04:48AM
(#30612697)
Well at least it’s better than numpy’s csv functions. For my use case (~4 million images, one row per image, 2048 fp32 columns for features extracted with resnet), parsing/encoding csv with pandas was 10x faster than with numpy. Of course serializing the arrays to/from npy was 10x faster than any csv library.
[ Parent
| on HN ]
There are no comments on this comment yet.