Spark is written in Scala, not Python. I've never used the Python interface, but I've extensively used Spark from Scala and while it's a little annoying to work with sometimes, for "real" Big Data (data set > 1TB maybe), its probably the best tool around.
Re:ND4J: Scientific Computing on the JVM
by stochastic_monk on Saturday May 18, 2019 @06:11PM
(#19950014)
Have you looked at Thrill? [0] It claims to basically be a better, C++ Spark.
Madness.
I am still aghast. How does anyone get any work done using those tools?
https://www.datacamp.com/community/tutorials/apache-spark-py...
Why didn't they implement RowSet interface?
Or create new tabular data holding interface?
Any way. Whenever someone enthuses about Python, Spark, big data, whatever, I now assume they're insane.
--
The whole notebook (Jupyter) idea is awesome. Are the fans conflating different tools, equating Spark with notebooks?