News for hackers. Stuff that matters.
Re:Dataflow/Beam and Spark: A Programming Model Comparison by chimerasaurus on Wednesday February 03, 2016 @05:29PM (#11030382)
Since Cloud Dataproc is ephemeral it's going to be a (probably) better idea to use GCS over HDFS so there is no data loss.

Technically, Cloud Dataproc clusters have both HDFS (on PD) for write/read-intensive operations (and scratch space) along with the GCS connector. GCS is not the default file system, however.

There are no comments on this comment yet.