The reason you don't bump into k8s while selling to Hadoop users is that Hadoop isn't something you'd run on a container-based stack (at least not right now and IMO it wont be). There are lots of Hadoop users who run containers for their application infra (as opposed to data infra). Pachyderm's whole pitch is that containerized data infra can be really powerful and that enterprises will want to unify their stack to all be containerized and k8s is THE answer for the orchestration layer.
P.S. Despite all my opinions above, I actually agree with your initial question around who zetcd is actually targeting. I don't have a clear picture of that.
Doesn't Pachyderm predate K8s though? Is this a recent development? Have they shifted focus then?
I will try to reply in the morning in depth on the other points.
What I'm trying to gauge here is k8s as an actual "company wide platform". I would love for it to be something I can depend on to be at an enterprise in a few years. It's great technology but still feels like it needs to be beaten up a bit yet.
You'd be surprised how many services you interact with on a daily basis are running on k8s (whole or in part).
It's still early, and many of the adopters today in large companies just happened to be making modernization efforts of their app-dev / app-deploy pipelines and moved to k8s or OpenShift. That said, it's certainly not ubiquitous yet.
I'd also never pick kubernetes for my "from scratch" cluster due to already being reliant on the JVM stack. I actually like the idea of giving an IT department that already understands zookeeper a mesos cluster with DC/OS.
That being said - k8s has a ton of momentum but it seems to be mainly with startups or maybe niche teams (prove me wrong here?) outside of google. It would be great to understand what you guys are looking at for things like this. Right now it feels like k8s and a lot of the other startups in this space like pachyderm are trying to compete with the hadoop ecosystem (which is great! competition forces innovation which is good for the ecosystem as a whole)