> that local kubernetes clusters or even bare metal clusters are hard
what's so hard about k3s, kubespray or minikube? the only problem k8s has, that without a linux dev machine you are going through the vm pain. but anything else is just simple.
btw. using mac sucks when trying to develop with local k8s. windows is better/okish, but it's best to run directly on linux.
Re:Why Does Developing on Kubernetes Suck?
by cassianoleal on Friday August 23, 2019 @09:01AM
(#20777035)
Why is it worse on the mac than on Windows? Legit question here, I haven't run Windows in many years so wouldn't know.
I find developing on k8s on the mac just fine. Most of the time I either just run the Docker for Desktop k8s feature, or fire up a kind cluster.
kind fits really well with VS Code's Remote - Containers extension for a completely isolated, all batteries included dev experience.
Year old experience: minikube requires root if you want to run directly on the host, without a VM layer. Every couple of months it goes into a borked state, which is nigh impossible to debug and sometimes hard to recreate from scratch, possibly due to the root issue above. Got a little luckier with Ubuntu and microk8.
thank you. As for #4, i've heard good things about https://istio.io - supposedly it does provide nice means to gain introspection to the traffic between services.
One part that I agree with the author is that local kubernetes clusters or even bare metal clusters are hard. I love being able to just spin up a cluster in GKE and have it just work. It really does just work, especially for ingress and storage. Get one piece working at a time, then you can move it to its final location and do whatever else needs to happen to make that work. It's valuable to have it working in the cloud even if prod is not, since that makes for easier testing/CI.
As for your #4, it is hard, although I've seen some success with running a pod that has tcpdump in it as a sidecar. Example here: https://medium.com/@xxradar/how-to-tcpdump-effectively-in-ku... (haven't used these directions personally, but it looks very similar to what I've seen). Some networking problems will only happen in kubernetes, things like timeout configuration and nginx stuff will just be different, so it's good to be able to do this.
One other note is for logs, just use the helm charts to install kibana/fluentd/elasticsearch into your cluster. It's really easy, and then you don't have to worry about which pod is making the logs, and by default it is tagged with the pod name and other k8s metadata.