Using Spark and Zeppelin to Process Big Data on Kubernetes 1.2

Re:Using Spark and Zeppelin to Process Big Data on Kubernete... by TheIronYuppie on Wednesday March 30, 2016 @05:07PM (#11393012)

Disclosure: I work at Google on Kubernetes.

If you'd like to do this in containers/Kubernetes, we'd love to highlight your work! Kubernetes runs great on AWS (as well as GCP, Azure and elsewhere), so no cloud migration required.

[ Parent | on HN ]

Re:Using Spark and Zeppelin to Process Big Data on Kubernete... by jaz46 on Wednesday March 30, 2016 @06:21PM (#11393492)

Shameless plug: Pachyderm is another way to run big data workloads in Kubernetes. github.com/pachyderm/pachyderm. We spoke at KubeCon SF last year and v1.0 is coming out next month!

[ Parent | on HN ]

Re:Using Spark and Zeppelin to Process Big Data on Kubernete... by TheIronYuppie on Wednesday March 30, 2016 @06:43PM (#11393600)

Disclosure: I work at Google on Kubernetes

Great to hear, congrats on reaching 1.0! Please do reach out when you get there, we'd love to show off your work to the community.

[ Parent | on HN ]

Re:Using Spark and Zeppelin to Process Big Data on Kubernete... by mindcrime on Wednesday March 30, 2016 @05:25PM (#11393150)

Awesome, we'll definitely be looking into that as we progress. "Version 1" is going straight to raw VM's as we saw it as the path of least resistance to getting an MVP out (esp. since nobody here has done much work with containers yet) but the container route is definitely something we'll be pursuing at some point.

[ Parent | on HN ]