News for hackers. Stuff that matters.
Re:Modern Python Performance Considerations by mritchie712 on Thursday May 05, 2022 @10:38AM (#31273935)
pandas
Re:Modern Python Performance Considerations by SnooSux on Thursday May 05, 2022 @10:39AM (#31273946)
It's been done: https://github.com/pola-rs/polars

But I'm sure there's always room for improvement

Re:Modern Python Performance Considerations by rytill on Thursday May 05, 2022 @10:58AM (#31274212)
It’s not like polars is a drop-in replacement, it has a totally different API.
Re:Modern Python Performance Considerations by mrtranscendence on Thursday May 05, 2022 @01:29PM (#31276202)
You wrote "it has a totally different API", did you mean "it has an actually sane API?" Because that's what I think of when I compare pandas to polars.
Re:Modern Python Performance Considerations by ies7 on Friday May 06, 2022 @12:29AM (#31281330)
Not if you've legacy apps using pandas everywhere.

Same API means I can import polars as pd and be done with it.

Re:Modern Python Performance Considerations by w-m on Thursday May 05, 2022 @12:47PM (#31275655)
This is a curious reply for me. I would think that there are very few parts in pandas that could be sped-up by reimplementing them with a compiled language. Pandas is plenty fast for the built-in methods, it only gets slow when you start interfacing with Python, e.g. by doing an `.apply` with your custom Python method. Obviously this interfacing part is impossible to speed up by reimplementing parts of pandas (you'd need a different API instead).
Re:Modern Python Performance Considerations by curiousgal on Thursday May 05, 2022 @01:24PM (#31276137)
I remember, when trying to squeeze some performance out of it, that a lot of the overhead came from it trying to infer types.
Re:Modern Python Performance Considerations by mynameis_jeff on Thursday May 05, 2022 @06:23PM (#31279035)
Re:Modern Python Performance Considerations by fgh on Thursday May 05, 2022 @10:39AM (#31273950)
The answer would then be to have a look at polars.