News for hackers. Stuff that matters.

A comment on Polars Cloud: The Distributed Cloud Architecture to Run Polars Anywhere. Parent comment

Re:Polars Cloud: The Distributed Cloud Architecture to Run P... by briankelly on Friday March 07, 2025 @07:48PM (#43296460)
Scala Spark - a bit absurd if you don't need the parallelism, though. Most of the development can be done simply in quick compilation iterations or copied from the sbt REPL. Python/pandas feels Stone Age in comparison - you absolutely waste a lot of time iterating with run-time testing.
[ Parent | on HN ]
Re:Polars Cloud: The Distributed Cloud Architecture to Run P... by Centigonal on Friday March 07, 2025 @08:39PM (#43296722)
Why scala spark over pyspark?
[ Parent | on HN ]
Re:Polars Cloud: The Distributed Cloud Architecture to Run P... by smu3l on Friday March 07, 2025 @08:51PM (#43296789)
Scala (and Java) has a typed Dataset api.[0] pyspark only provides the Dataframe API, which is not typed.

[0] https://spark.apache.org/docs/latest/sql-programming-guide.h...

[ Parent | on HN ]
Re:Polars Cloud: The Distributed Cloud Architecture to Run P... by Centigonal on Saturday March 08, 2025 @10:55AM (#43301133)
thanks!
[ Parent | on HN ]