samnco
Simple Perf Test for Tensorflow Serving on K8s
OK I gave it a try and you are absolutely right. For the nvidia-smi, I could run it the /dev/nvidia0, which is cool.
I was also able to run it unprivileged. I guess my mistake was to believe the example from the docs and not test without.
Thanks for sharing that, I'll update my charts and the post accordingly.
By default, the bundle come with a "auto" tag, which will activate privileged containers just when GPUs are detected.
You can enforce "false" to remove that, but then you won't be able to run GPU workloads.
Or you can enforce "yes" and have them activated all the time.
Does that answer the question? Not sure if I understood it right.
Replicating is not very hard. You need a lightweight x86 machine for MAAS, which takes ~20min to install, one VLAN for the iDRAC (IPMI), another for networking that can connect to internet, and off you go. You can also enable KVM power management in MAAS to run the Juju control plane in VMs and save a box if you're limited in compute power.
https://maas.io https://jujucharms.com/docs https://www.ubuntu.com/containers/kubernetes for all the goodies.
If you run into problems, I am SaMnCo on #juju in freenode.
So many potential failure points in there. The sole use case is CUDA. Essentially I wanted a portable cluster with GPUs and that did the work for a couple of month. Now it's getting more serious so the switch to T630 makes sense, and I repurposed the NUCs into the control plane of the K8s cluster.
The PSU is the Corsair AX1500i (1500W), with 10x lines for GPUs. It's robust on paper, didn't have any problem with just 4 plugged in.
But I must say... The T630 are very noisy compared to these, but so much more powerful #NotGoingBack
It works, but the GPUs aren't very stable at 4x vs. a normal 16x.
At this stage, you have no data. Normally, if you app depends on data, the pods will keep failing until data has been moved and is available (at which point they stabilize in an equilibrium state).
If you run Ceph in both environments, then you may want to use Ceph replication. If you use another storage layer, then it's also up to you to make sure the data is moved around.
rook should have support for that use case. However, it is still alpha as per the GitHub readme, so use at your own risk. However, you may want to consider the data replication problem outside of the scope of k8s.
If the 2 sets of clusters are physically close to each other, you may want to just point the new apps to the old data and pull the switch from the first one.
Another option would be to run another beta feature in K8s called Federation, which allows to manage several cluster via a single control plane.
Sorry for the long comment, your question has a broad scope, and it's hard to answer without diving into details.
Canonical at this stage only supports Ceph commercially, but it doesn't mean GlusterFS is not a good option. I haven't tried it myself, so can't tell.
Anyone?
In general and in my opinion, it is always better to run with k8s since you have, for the stateless pieces, cluster awareness. So there is never a downside to it, especially as the control plane is very lightweight for small clusters, and you can colocate many parts.
* Run Ceph in separate nodes and connect it to the cluster. With Juju, you can do that from the bundle, as Ceph is also a supported workloads. This gives you scale for storage
* Run Ceph within the cluster with a Helm chart. We see that for openstack-helm for example. Also gives you scale, but the lack of device discovery makes it less interesting in my opinion
* Run an NFS server, plain easy but not very scalable.
* Use hostpath, which is the default but doesn't get you scale.
Another use case is in media for transcoding. It is not a trivial job to orchestrate transcoding at scale, and Kubernetes with or without GPUs is an excellent solution for that as it is trivial to setup a completely automated job queue.
Also another interesting field will eventually be HPC but there are some constraints about compute that K8s does not tick scheduling wise at this point in time. There is a pluggable scheduler in the works I think, and this will eventually help. Also the LXD example is a nice optimization but it would not replace the scheduler in any way.
[0]: https://medium.com/intuitionmachine/gpus-kubernetes-for-deep...
[1]: https://hackernoon.com/job-concurrency-in-kubernetes-lxd-and...
* self healing: when you create a deployment/replica set. it will be maintained at all cost, so if the app has a memory leak or anything goes wrong, it will be contained and kept up and running
* Rolling update: even when you run 5 frontends, it is a pain to use capistrano or other tools to just update at git repo. it is literally a one liner in Kubernetes. If you use CI/CD the setup is just a few lines in any Jenkins/Gitlab/Travis...
* Service discovery: the combination of ENV and predictable DNS endpoints is just awesome
* Ecosystem: PaaS, Serverless... Many of the new world infra is built on K8s, so it is a door to the next gen, whether you know you will use it or not.
As for Micro Service Architecture, just starting with the web frontend and a couple of lightly dockerize middleware makes it sooooo simple that you instantly want to get more out of it.
As the overhead of running K8s vs. set of servers is relatively low especially at small scale, it is definitely worth looking at. Happy to do a run through with you and show you how the deployment of a tiered app works as a demo, ping me on @SaMnCo_23 if interested.