kommissar
This is spot on IMO. I use Glue internally (opinions are my own) and still believe that the best course of action is Glue should only run managed Spark. We provide an empty Scala "script" that does nothing, and load a compiled JAR file with the Scala code that actually runs our job as a library and have Glue exec into that.
We can version the ETL in git, run local tests outside of the Glue data plane, prototype in the Spark shell, and much more.
I'm still not sure what this is. Who is ec2spot??
My guess is that maybe Amazon installed the Folding client on some part of ec2 capacity in an idle pool waiting to be launched?
I'm not sure. That would use quite a lot of power... and setting up networking and getting approval to do this would be a ton of work.