Re:Apache Arrow and the “Things I Hate About Pandas”
by ves on Tuesday September 26, 2017 @12:29AM
(#15336176)
We use pandas as a last-mile library for offline exploration. Typically, datasets have been sampled or aggregated enough that they can fit inside 10gb so you can work with them comfortably using pandas. I don't like using pandas in prod because the performance is really sensitive to stuff like missing a type declaration or calling the wrong method and the API is really convoluted.
E: that's not to say pandas isn't good. It's really good. Thanks for the software, Wes!
E: that's not to say pandas isn't good. It's really good. Thanks for the software, Wes!