News for hackers. Stuff that matters.
Re:Should You Ditch Spark for DuckDB or Polars? by nicornk on Sunday December 15, 2024 @01:59PM (#42425213)
The blog post echos my experience that duckDB just works (due to superior disk spilling capabilities) and polars OOMs a lot.
Re:Should You Ditch Spark for DuckDB or Polars? by jdgoesmarching on Sunday December 15, 2024 @02:52PM (#42425519)
My biggest issue with Polars (don’t shoot me) was the poor LLM support. 4o at least seems to frequently get confused with Pandas syntax or logic.

This pushed me to finally investigate the DuckDB hype, and it’s hard to believe I can just write SQL and it just works.

Re:Should You Ditch Spark for DuckDB or Polars? by serjester on Sunday December 15, 2024 @06:16PM (#42426648)
The API has mostly stabilized and at this point other than some minor errors (“groupby” vs “group_by”) LLM’s seem to do pretty well with it. Personally, I’m glad they made the breaking changes they did for the long-term health of the library.
Re:Should You Ditch Spark for DuckDB or Polars? by diamondfist25 on Monday December 16, 2024 @01:34AM (#42428518)
Problem is ur using 4o. Use 3.5 sonnet, its much better
Re:Should You Ditch Spark for DuckDB or Polars? by jamesblonde on Monday December 16, 2024 @02:04AM (#42428676)
groupby and group_by is a big problem for polars with 4o.