It's especially interesting because of all of the other implications of moving to Spark — like trying to debug your app code that runs PySpark that runs Scala Spark on the JVM. It's very difficult to trace errors all the way through, no way to use tools like pdb. It takes me so much longer to debug errors in Spark code.