News for hackers. Stuff that matters.
Re:Polars by mjhay on Tuesday January 09, 2024 @09:52AM (#38926827)
Laziness in this context has huge advantages in reducing memory allocation. Many operations can be fused together, so there's less of a need to allocate huge intermediate data structures at every step.
Re:Polars by disgruntledphd2 on Tuesday January 09, 2024 @10:30AM (#38927241)
yeah, totally, I can see that. I think that polars is the first library to do this locally, which is surprising if it has so many advantages.
Re:Polars by mjhay on Tuesday January 09, 2024 @10:57AM (#38927549)
It's been around in R-land for a while with dplyr and its variety of backends (including Arrow, the same as Polars). Pandas is just an incredibly mediocre library in nearly all respects.
Re:Polars by disgruntledphd2 on Wednesday January 10, 2024 @05:49AM (#38938790)
> It's been around in R-land for a while with dplyr and its variety of backends

Only for SQL databases, so not really. Source: have been running dplyr since 2011.

Re:Polars by mjhay on Friday January 12, 2024 @10:59AM (#38969618)
The Arrow backend does allow for lazy eval.

https://arrow.apache.org/cookbook/r/manipulating-data---tabl...