Modern Polars: an extensive side-by-side comparison of Polars and Pandas

Re:Modern Polars: an extensive side-by-side comparison of Po... by brahbrah on Monday January 09, 2023 @09:18AM (#34310555)

By any chance were you iterating over your pandas dataframe or using .apply? I’d be surprised by any properly formatted (i.e. vectorized) pandas operation that takes that long for data that fits in memory

[ Parent | on HN ]

Re:Modern Polars: an extensive side-by-side comparison of Po... by mutant_self on Monday January 09, 2023 @09:40AM (#34310808)

Here's an example of idiomatic Pandas taking 10 minutes while Polars takes 7 seconds: https://www.pola.rs/posts/the-expressions-api-in-polars-is-a...

[ Parent | on HN ]

Re:Modern Polars: an extensive side-by-side comparison of Po... by brahbrah on Monday January 09, 2023 @09:54PM (#34319903)

I'm not saying that polars isn't faster. In fact in my other comment here I mention that polars is much better than pandas at what polars does (it's not a drop in replacement). I'm just saying that most of the times (not always, and in fact in those cases we've used polars to speed it up) that I've seen painfully slow pandas operations has been due to poorly formatted pandas code.

[ Parent | on HN ]