News for hackers. Stuff that matters.
Re:It’s Time to Embrace Kubernetes Really? by jaxxstorm on Thursday December 10, 2020 @02:19PM (#25377330)
Hi everyone! Article author here!

My friend and colleague, Paul Stack: https://twitter.com/stack72 are at very opposite ends of the spectrum when it comes to Kubernetes. We put our heads together and tried to create a balanced take on Kubernetes where our opinions didn't cloud the article. It's obviously very polarizing, but I think we managed to ask some good questions before you started the adoption process.

Would love to hear more via twitter too! https://twitter.com/briggsl

Re:It’s Time to Embrace Kubernetes Really? by area51org on Thursday December 10, 2020 @02:26PM (#25377434)
Thx for writing that. To me it seems mind-boggling that in the 21st century you still have to point out to people that you shouldn't just adopt a technology because it seems cool. Of course you should ask yourself what problem you're trying to solve. Apparently people still don't get this.

I'm not sure I agree about templating YAML files–why is templating JSON any better, really?–but YMMV.

Re:It’s Time to Embrace Kubernetes Really? by hpoe on Thursday December 10, 2020 @02:43PM (#25377646)
I agree that technology shouldn't be adopted just because it is new unfortunately the job market makes it so many early engineers (and already established engineers that will want to move at some point) end up practicing RDD as a matter of self interest.
Re:It’s Time to Embrace Kubernetes Really? by rektide on Friday December 11, 2020 @11:46PM (#25395645)
there's basically no other options for running your own cloud & there's no particular major weak points for Kubernetes that other containerizing automation engines don't also have.

people think they're being smart by telling you to check whether such and such fits your needs. usually it's not bad advice. but the deck is pretty stacked here. nothing else has shown itself to be as flexible, as extensible, as consistent. well demonstrated but it's capability at handling many different add-ons (controllers, operators). rarely is cloud architecture do readily deployable to other non-core concerns, especially via the core tools & constructs.

if you want to go march off in to tech that does less, that will only do a little bit, that less people will be familiar with, that has less community, sure. spend a couple hours justifying to yourself why you are not going to use the really amazing all encompassing tech that's quickly supplanted everything else. pick some other combination of technologies which you will assemble, make your own stack, yeah, you're an independent thinker, you don't need convention or all this extra stuff! it's just gonna slow you down. go for it.

Kubernetes is not easy. but I find arguing against it to be difficult. there have been tools upon tools upon tools for so long. but little out there has brought the different concerns together, found overarching paradigms to create a basic, trusted quality of life that Kubernetes has clarified. again, it's not easy, but learning it ought to keep paying forward again & again, as you pick up new interests, want new services. even if those are Amazon services, ACK now will let you build Kubernetes deployment units that include those AWS services. the paradigm works.

Re:It’s Time to Embrace Kubernetes Really? by lyptt on Thursday December 10, 2020 @02:08PM (#25377177)
One thing I've found about k8s is running a cluster can be very costly, and using a managed cluster on one of the big three clouds is even more so. Also, when rolling your own cluster it requires a lot of elbow grease to get things in a working state, and even more to keep things working.

It feels awesome from a developer standpoint once you work out all the kinks. It's very natural describing your service with a bit of yaml, and then being able to then spawn it into your infrastructure and have it automatically scale out.

If there was a cost effective k8s cluster you could rent then that would be the thing that finally kills traditional infrastructure for me.

Re:It’s Time to Embrace Kubernetes Really? by rhacker on Thursday December 10, 2020 @02:12PM (#25377217)
That's the exact same feeling I get too. If I'm a mom/pop shop online, why can't I get a k8s cluster w/just 1 machine for $5/month? I mean the big three (4 w DO) can all get you a bare machine for $5/mo that can run Docker. Why not give the k8s management side for free for those people?
Re:It’s Time to Embrace Kubernetes Really? by algeoMA on Thursday December 10, 2020 @02:19PM (#25377324)
You can get a small cluster from Digital Ocean pretty cheaply. It's more than $5 (I forget the exact price) but they'll provide a free master node and then you just pay for one additional kubelet (and you can add more as you need them). I have used it for one hobby project and have been pretty happy with it.
Re:It’s Time to Embrace Kubernetes Really? by geerlingguy on Thursday December 10, 2020 @02:40PM (#25377593)
Same with Linode, the management machines are not massive scale HA control planes like you get with the expensive hosts (It seems like Linode may run a single 2GB droplet), but for most of my clusters I don't need it.
Re:It’s Time to Embrace Kubernetes Really? by rhacker on Thursday December 10, 2020 @02:27PM (#25377452)
I'll have to check that out!
Re:It’s Time to Embrace Kubernetes Really? by OJFord on Thursday December 10, 2020 @02:15PM (#25377257)
Because that's more the realm of:

https://www.digitalocean.com/products/kubernetes/

https://www.scaleway.com/en/kubernetes-kapsule/

IaaS model is more 'everything is chargeable in small units', a free control plane doesn't really gel with that IMO, and also they have some huge users that it'd cost a lot to run the control plane for, pushing up the price of what you connect to it (or something must give) and making more PaaS-style like DO & Scaleway anyway.

Re:It’s Time to Embrace Kubernetes Really? by throwaway201103 on Thursday December 10, 2020 @02:14PM (#25377240)
If you're mom/pop online shop and just need 1 machine you don't need k8s. KISS principle applies.
Re:It’s Time to Embrace Kubernetes Really? by WJW on Thursday December 10, 2020 @02:22PM (#25377366)
From my experience, deploying to a k8s cluster managed by someone else is extremely simple and straightforward. Tell it which container to run and off it goes. You get zero-downtime rolling deploys, automatic restarts on crashes, easy rollbacks, service discovery (if you have multiple services) and more for free.

Managing a cluster is the tricky part of k8s but just running your stuff on an existing cluster is extremely straightforward. I could definitely see the attraction of a cheap managed k8s cluster with a single node. Maybe hosted k3s or something.

Re:It’s Time to Embrace Kubernetes Really? by bpodgursky on Thursday December 10, 2020 @02:19PM (#25377327)
A ton of the value from k8s is in the programmatic and replicable configuration. 1 computer pets are still bad in a mom and pop shop. 1 computer livestock is still better.
Re:It’s Time to Embrace Kubernetes Really? by kitotik on Thursday December 10, 2020 @02:46PM (#25377682)
Is “1 computer livestock” even possible?

After your single cattle is murdered, the next one has to be raised right?

State has to come from somewhere. Downtime has a cost for most businesses.

Re:It’s Time to Embrace Kubernetes Really? by aaomidi on Thursday December 10, 2020 @04:06PM (#25378669)
In this case the state can come from a centrally managed k8s controller. Something that most cloud providers provide and have good SLAs on.

Kubernetes just makes that story so much simpler. Heck a good cloud provider can just handle any machines fucking up automatically without any human intervention on your side.

The same can't be said for a single machine.

Re:It’s Time to Embrace Kubernetes Really? by bpodgursky on Thursday December 10, 2020 @03:18PM (#25378023)
Downtime has cost.

But being able to blow away a k8s node and bring it back up in 2 minutes (or have k8s auto-spawn a replacement node) is WAY faster than having to reinstall wordpress, reinstall nginx, set IP addresses, fiddle with your DNS, restore a database from a backup, etc, etc, etc.

Re:It’s Time to Embrace Kubernetes Really? by throwaway894345 on Thursday December 10, 2020 @02:25PM (#25377417)
Not to mention everything you have to do to get a machine to parity with Kubernetes--set up SSH, process management, monitoring, logging, etc. I don't think I'd go so far as to say Kubernetes is the right tool for this use case, but I certainly empathize with the sentiment.
Re:It’s Time to Embrace Kubernetes Really? by twelve40 on Thursday December 10, 2020 @04:44PM (#25379146)
so... for a small cluster then it's like aws beanstalk, right? rolling deployments, load balancing, etc. all for free.
Re:It’s Time to Embrace Kubernetes Really? by rhacker on Thursday December 10, 2020 @02:18PM (#25377296)
Well, in this case the pop is me, and I know k8s but I don't want to invest 15-35 / month on a side project but would still rather have a decently managed site rather. I don't want to go back to FTP crap at this point.

A car is a hugely complicated machine and I'm sure in the 1910s people said only some people will ever need them. Bad analogy considering cars kill lots of people but take that part out :)

Re:It’s Time to Embrace Kubernetes Really? by dcolkitt on Thursday December 10, 2020 @02:21PM (#25377353)
You can run a single-core preemptible instance on GKE for $7/month. As long as you keep the one instance alive that persists the cluster.
Re:It’s Time to Embrace Kubernetes Really? by renewiltord on Thursday December 10, 2020 @02:32PM (#25377518)
Get the zonal cluster on GKE. Free control plane stuff. Then you can put as many or as few nodes as you want on it.

Make sure to use metal LB or all your ingresses will be served with Google LBs that are hecka expensive.

Re:It’s Time to Embrace Kubernetes Really? by comradesmith on Thursday December 10, 2020 @02:23PM (#25377383)
You can with digital ocean
Re:It’s Time to Embrace Kubernetes Really? by area51org on Thursday December 10, 2020 @02:27PM (#25377448)
And, not everything should be in a container. Just because you can does not necessarily mean you should. Containers are awesome but they are not a cure-all silver bullet.
Re:It’s Time to Embrace Kubernetes Really? by flowerlad on Thursday December 10, 2020 @02:27PM (#25377446)
> One thing I've found about k8s is running a cluster can be very costly, and using a managed cluster on one of the big three clouds is even more so.

I run a 3-node Kubernetes cluster on Google cloud. It costs me around $100 per month (with 1-year reserved instances). Not terribly expensive.

Re:It’s Time to Embrace Kubernetes Really? by crb002 on Friday December 11, 2020 @12:43PM (#25388407)
What is your utilization? IMHO I would rather do one EC2 instance then have EC2 Batch spot and lambda workers take the other load. In a good design most media assets should never hit the full time provisioned server.
Re:It’s Time to Embrace Kubernetes Really? by k__ on Thursday December 10, 2020 @02:20PM (#25377343)
Haha.

I forgot to tear down a small EKS cluster last month. 4 days, 60€.

Re:It’s Time to Embrace Kubernetes Really? by alexfromapex on Thursday December 10, 2020 @02:01PM (#25377095)
There should be more of a boilerplate very stable version for smaller companies but I think in general Kubernetes makes it really easy to automate away a lot of the ops and security stuff and get app pipelines running pretty quickly which is really nice.
Re:It’s Time to Embrace Kubernetes Really? by vorpalhex on Thursday December 10, 2020 @02:41PM (#25377618)
There are companies trying to provide this - I don't know if anyone is doing it very successfully yet, but Microk8s, k3s, etc are all basically "distros" of kubernetes for different targets.

I think part of the issue is that many of the bigger players want you to use their managed k8s solution and not just give you their setup - I think that's a temporary thing. A few of the big boons of k8s is the ability to be multi-cloud, and vendors are losing out on a revenue stream by not embracing that (though, AWS has a new multi-cloud k8s thing, so maybe they get it).

Re:It’s Time to Embrace Kubernetes Really? by nickthemagicman on Thursday December 10, 2020 @02:03PM (#25377126)
Managed K8's right?
Re:It’s Time to Embrace Kubernetes Really? by p_l on Thursday December 10, 2020 @02:36PM (#25377551)
Not only managed options.

Openshift is big in on-prem enterprise setups because as flawed as the install process is or as ridiculous licensing is, you get a remarkably complete system pretty easily out of a bunch of servers that integrates with things like your corporate LDAP.

Re:It’s Time to Embrace Kubernetes Really? by politelemon on Thursday December 10, 2020 @02:09PM (#25377183)
I found this to be actually quite a well balanced take. There are quite a lot of useful considerations in here which I don't often see in HN discussions about k8s, as it's usually from people well invested in that ecosystem now.

Some of our teams have been drifting naturally towards ECS Fargate and found it to be a great fit. Almost invariably the k8s question comes up, and it's usually a reaction to the popularity of k8s and EKS. It's quite easy to start doubting your own well thought-out decisions in the face of a large popular elephant sitting in your rooms which wants to trample on everything.

Re:It’s Time to Embrace Kubernetes Really? by threeseed on Thursday December 10, 2020 @03:00PM (#25377827)
You can also get EKS Fargate so best of both worlds.

ECS is definitely more suited for small use cases however.

Re:It’s Time to Embrace Kubernetes Really? by tyingq on Thursday December 10, 2020 @02:31PM (#25377507)
Unfortunately, the hoopla is loud enough that developers could reasonably be expected to push for it just so they could have real experience to put on their resumes.
Re:It’s Time to Embrace Kubernetes Really? by nickthemagicman on Thursday December 10, 2020 @02:03PM (#25377123)
I kind of don't understand why people don't use managed K8's and avoid needing to pay ops people to manage their setup.

One can learn the developer side of K8's in a week or so enough to be deploying apps and setting up services and pods without the overhead of managing etcd and distributed clusters.

Re:It’s Time to Embrace Kubernetes Really? by Townley on Thursday December 10, 2020 @02:20PM (#25377333)
I use managed k8s, but have sympathy for the bare metal folks. Wiring servers together using kubeadm is so simple and powerful that the missing pieces of the puzzle (networking and storage, for example) feel frustratingly close.

In an ideal world, you could purchase or rent commoditized resources (compute, storage, bandwidth) and have those resources deploy your application in a standardized way. K8s is so close to that out of the gate, that having to use cloud provider's expensive and proprietary offerings feels sub-optimal.

I agree that cloud offerings are the way to go for anyone without good reasons not to (eg compliance, lack of connectivity, or full control over upgrades). But a lot of the "I prefer on prem" attitude comes from realizing that having a black box of an ingress controller on my cluster isn't the end state of how apps get deployed

Re:It’s Time to Embrace Kubernetes Really? by throwaway201103 on Thursday December 10, 2020 @02:13PM (#25377229)
When your site or app goes offline it can be handy if you have ops people who understand your setup.
Re:It’s Time to Embrace Kubernetes Really? by OJFord on Thursday December 10, 2020 @02:21PM (#25377363)
If EKS goes down it doesn't help its users nor AWS for its ops people to know how your webapp's configured.
Re:It’s Time to Embrace Kubernetes Really? by area51org on Thursday December 10, 2020 @02:31PM (#25377504)
Setup = a k8s service? You should have support from the vendor, period.

Setup = rep groups, internal services, etc? You need k8s-savvy ops people, or really good devs who like ops stuff.

Re:It’s Time to Embrace Kubernetes Really? by threeseed on Thursday December 10, 2020 @03:09PM (#25377931)
> You need k8s-savvy ops people, or really good devs who like ops stuff

They are called DevOps Engineers and it's a standard role these days.

And it doesn't matter whether they like it or not. In many organisations these days developers own their piece of work all the way through to production and are expected to provide an ops-like support for it.

Re:It’s Time to Embrace Kubernetes Really? by vorpalhex on Thursday December 10, 2020 @02:43PM (#25377643)
Managed k8s doesn't mean you don't need ops people. You still need ops people - you are still running a complex and customized multi-node kubernetes cluster. You just need fewer ops people by following a vendors optimistic path.
Re:It’s Time to Embrace Kubernetes Really? by area51org on Thursday December 10, 2020 @02:30PM (#25377490)
It really depends on what your needs are and whether your provider's k8s service addresses them. AKS for example does work but, I'm told, is missing some fundamental stuff in regards to security etc. (Caveat: I only know this through others.)

OTOH if a service is available and would work for someone's use case, by all means, use the service. I've set up native k8s clusters just as exercises, and it's no picnic–and I didn't need to manage those clusters, later. Just doing build exercises alone was enough to scare me straight.

Re:It’s Time to Embrace Kubernetes Really? by dilyevsky on Thursday December 10, 2020 @02:29PM (#25377471)
For prod? You’ll know when your gke control plane goes down and support is twiddling their thumbs for two days
Re:It’s Time to Embrace Kubernetes Really? by area51org on Thursday December 10, 2020 @02:32PM (#25377520)
Does that actually happen with GKE? If so: do not use GKE. Other vendors surely have better-supported k8s services.
Re:It’s Time to Embrace Kubernetes Really? by dilyevsky on Thursday December 10, 2020 @02:40PM (#25377592)
Yes at previous gig someone took down gke control plane by creating many crd objects (kubeflow) and it took multiple escalations to fix.
Re:It’s Time to Embrace Kubernetes Really? by dilyevsky on Thursday December 10, 2020 @02:41PM (#25377617)
Also fwiw i heard similar stories about aks and eks as well as do
Re:It’s Time to Embrace Kubernetes Really? by tcoff91 on Thursday December 10, 2020 @02:33PM (#25377533)
2 days of downtime from a GCP service? I didn't know the outages with these cloud services were ever that bad.
Re:It’s Time to Embrace Kubernetes Really? by dilyevsky on Thursday December 10, 2020 @02:40PM (#25377605)
Two days is nothing. I had their fabric black holing packets for weeks before it got fixed
Re:It’s Time to Embrace Kubernetes Really? by nickthemagicman on Thursday December 10, 2020 @03:35PM (#25378255)
This could be said about ANYthing in the cloud though
Re:It’s Time to Embrace Kubernetes Really? by dilyevsky on Thursday December 10, 2020 @03:37PM (#25378270)
Absolutely, anything vendored out even.
Re:It’s Time to Embrace Kubernetes Really? by nickthemagicman on Friday December 11, 2020 @11:42AM (#25387678)
This could be said of using the cloud in general though.

But people still use the cloud.

Re:It’s Time to Embrace Kubernetes Really? by crb002 on Thursday December 10, 2020 @06:34PM (#25380264)
What devs really want:

* Easy simulation of networks between cgroups.

* A mechanism for splitting a ulimit across kernels.

* Hot swappable kernels and kernel tuning parameters so machines can be quickly re-configured.

Re:It’s Time to Embrace Kubernetes Really? by crb002 on Thursday December 10, 2020 @06:42PM (#25380334)
I think the answer is Linux on Linux. Fault Tolerant Linux on Distributed Machine Linux on traditional multicore Linux. Distributed machine Linux just sees the multicore Linux boxes as devices. Fault Tolerant Linux does reruns when things fail.
Re:It’s Time to Embrace Kubernetes Really? by ranjanprj on Thursday December 10, 2020 @02:16PM (#25377264)
check out fsKube a desktop web ui, it can help developers just starting out with k8s in provisioning, DevOps to Monitoring with just few clicks

https://github.com/rebataur/fskube

[dead]
Re:It’s Time to Embrace Kubernetes Really? by dang on Thursday December 10, 2020 @01:54PM (#25377017)
Please don't do this here.
Re:It’s Time to Embrace Kubernetes Really? by eyberg on Thursday December 10, 2020 @02:31PM (#25377502)
Seems more and more people are waking up to the fact that docker and k8s are broken by design:

https://thenewstack.io/unfixable-kubernetes-security-hole-me...

Re:It’s Time to Embrace Kubernetes Really? by __MatrixMan__ on Thursday December 10, 2020 @02:37PM (#25377567)
Just because you have to trust your roommates a little doesn't mean that the idea of roommates is broken by design. It just means that you should be careful who you sign a lease with.
Re:It’s Time to Embrace Kubernetes Really? by eyberg on Thursday December 10, 2020 @02:44PM (#25377662)
How do you be careful?

You can't trust the cloud provider:

https://unit42.paloaltonetworks.com/windows-server-container...

https://media-exp1.licdn.com/dms/image/C5622AQH70okwLkMlNg/f...

What about the concept of not having roommates? Plenty of infrastructure alternatives exist besides "containers".

Re:It’s Time to Embrace Kubernetes Really? by threeseed on Thursday December 10, 2020 @03:07PM (#25377897)
You understand how VPS/EC2 etc work. They are also multi-tenant architectures i.e. having roommates.

So in your world you have to only use bare metal ? Sounds fun.

Re:It’s Time to Embrace Kubernetes Really? by dragonwriter on Thursday December 10, 2020 @03:16PM (#25378003)
> You understand how VPS/EC2 etc work. They are also multi-tenant architectures

EC2 basic instances are multitenant, but there are such a thing as EC2 dedicated instances (and dedicated hosts), which run on single-customer reserved hardware.

Re:It’s Time to Embrace Kubernetes Really? by threeseed on Thursday December 10, 2020 @03:05PM (#25377885)
You have given the ability for a normal user to modify load balancers and then are surprised they can do things like intercept traffic.

It's a bug but a very minor one. Especially if you are securing traffic inside the cluster.