News for hackers. Stuff that matters.
Re:“Let’s use Kubernetes.” Now you have eight problems by rawoke083600 on Thursday March 05, 2020 @03:26AM (#22491881)
"Pod, Deployment, Job, CronJob, Service, Ingress, ConfigMap, Secre"

Wow as a new developer coming onboard your company, I will walk out the door after seeing that, and the fact that you admit its a small serivce.

Re:“Let’s use Kubernetes.” Now you have eight problems by jorams on Thursday March 05, 2020 @03:39AM (#22491942)
The names may seem daunting, but here's what they do: a Pod is one or more containers, usually 1, that run an application. A Deployment manages a number of the same pods, making sure the right number is running at all times and rolling out new versions. A Job runs a pod until it succeeds. A CronJob creates a job on a schedule. A service provides a DNS name for pods of a certain kind, automatically load balancing between multiple of them. An Ingress accepts traffic from the outside and routes it to services. A ConfigMap and a Secret are both just sets of variables to be used by other things, similar to config files but in the cluster.

It's a small service according to " web scale", but it's serving, and vital for, a good number of customers.

Re:“Let’s use Kubernetes.” Now you have eight problems by eska on Thursday March 05, 2020 @09:51AM (#22493935)
People seem to hate learning vocabulary for concepts that they are already dealing with. Personally I love having a common vocabulary to have precise conversations with other developers.
Re:“Let’s use Kubernetes.” Now you have eight problems by hedora on Thursday March 05, 2020 @10:22AM (#22494234)
It’s more than that. People hate learning concepts that should be abstracted away from then.

As an example, why can’t ConfigMap and Secret just be plain files that get written to a well known location (like /etc)?

Why should the application need to do anything special to run in kubernetes? If they are just files, then why do they have a new name? (And, unless they’re in /etc, why aren’t they placed in a standards-compliant location?)

If they meet all my criteria, then just call them configuration files. If they don’t, then they are a usability problem for kubernetes.

Re:“Let’s use Kubernetes.” Now you have eight problems by kube-system on Thursday March 05, 2020 @10:51AM (#22494562)
ConfigMaps and Secrets can hold multiple files, or something that isn't a file at all, and your application doesn't have to do anything special to access them. You define how they configure your application in the pod/deployment definition. They solve a problem not previously solved by a single resource.

Maybe you don't personally find value in the abstraction, but there are certainly people who do find it useful to have a single resource that can contain the entire configuration for a application/service.

Re:“Let’s use Kubernetes.” Now you have eight problems by curryst on Thursday March 05, 2020 @12:30PM (#22495776)
They can be files on the disk. They can also be used as variables that get interpolated into CLI args. What they really are is an unstructured document stored in etcd that can be consumed in a variety of ways.

As the other user said, they can also be multiple files. I.e. if I run Redis inside my pod, I can bundle my app config and the Redis config into a single ConfigMap. Or if you're doing TLS inside your pod, you can put both the cert and key inside a single Secret.

The semantics of using it correctly are different, somewhat. But you can also use a naive approach and put one file per secret/ConfigMap; that is allowed.

Re:“Let’s use Kubernetes.” Now you have eight problems by nrb on Thursday March 05, 2020 @04:15AM (#22492106)
It's an afternoon worth of research to understand the basic concepts. Then, with the powerful and intuitive tooling you can spin up your own cluster on your computer in minutes and practice deploying containers that:

- are automatically assigned to an appropriate machine(node) based on explicit resource limits you define, enabling reliable performance

- horizontally scale (even automatically if you want!)

- can be deployed with a rolling update strategy to preserve uptime during deployments

- can rollback with swiftness and ease

- have liveness checks that restart unhealthy apps(pods) automatically and prevent bad deploys from being widely released

- abstracts away your infrastructure, allowing these exact same configs to power a cluster on-prem, in the cloud on bare metal or vms, with a hosted k8s service, or some combination of all of them

All of that functionality is unlocked with just a few lines of config or kubectl command, and there are tools that abstract this stuff to simplify it even more or automate more of it.

You definitely want some experienced people around to avoid some of the footguns and shortcuts but over the last several years I think k8s has easily proven itself as a substantial net-positive for many shops.

Re:“Let’s use Kubernetes.” Now you have eight problems by scarface74 on Thursday March 05, 2020 @09:55AM (#22493967)
So why should I do all of that instead of throwing a little money at AWS, run ECS and actually spend my time creating my product?

Heck, if my needs are simple enough why should I even use ECS instead of just putting my web app on some VM's in an auto-scaling group behind a load balancer and used managed services?

Re:“Let’s use Kubernetes.” Now you have eight problems by Legogris on Thursday March 05, 2020 @10:28AM (#22494300)
I don't think anyone is arguing that you should use k8s for a simple web app. There's definitely some inherent stack complexity threshold before solutions like k8s/mesos/nomad are warranted.

When you start having several services that need to fail and scale independently, some amount of job scheduling, request routing... You're going to appreciate the frameworks put in place.

My best advice is to containerize everything from the start, and then you can start barebones and start looking at orchestration systems when you actually have a need for it.

Re:“Let’s use Kubernetes.” Now you have eight problems by scarface74 on Thursday March 05, 2020 @02:21PM (#22497069)
If you need to fail and scale independently.

-you can use small EC2 instances behind an application load balancer and within autoscaling groups with host based routing for request routing.

- converting a stand-alone api to a container is not rocket science and nor should it require any code rewrite.

- if you need to run scheduled Docker containers that can also be done with ECS or if it is simple enough lambda.

- the first thing you should worry about is not “containerization”. Its getting product market fit.

As far as needing containerization for orchestration, you don’t need that either. You mentioned Nomad. Nomad can orchestrate anything - containers, executables, etc.

Not to mention a combination of Consul/Nomad is dead simple. Not saying I would recommend it. In most cases (I’ve used it before), but only because the community and ecosystem is smaller. But if you’re a startup, you should probably be using AWS or Azure anyway so you don’t have to worry about the infrastructure.

Re:“Let’s use Kubernetes.” Now you have eight problems by geggam on Thursday March 05, 2020 @11:05AM (#22494725)
What requirement is driving containers ?

How are you managing your infrastructure and if you have that already automated how much effort is it to add the software you develop to that automation vs the ROI of adding another layer of complexity ?

The idea everything needs to be in containers is similar to the idea everything needs to be in k8s.

Let the business needs drive the technology choices don't drive the business with the technology choices

Re:“Let’s use Kubernetes.” Now you have eight problems by Legogris on Thursday March 05, 2020 @11:11AM (#22494784)
Portability - the idea being that you can migrate to an orchestration technology that makes sense when you have the need. The cost and effort of containerizing any single service from the get-go should be minimal. It also helps a lot with reproducibility, local testing, tests in CI, etc.

Valid reasons to not run containerized in production can be specific security restrictions or performance requirements. I could line up several things that are not suitable for containers, but if you're in a position of "simple but growing web app that doesn't really warrant kubernetes right now" (the comment I was replying to), I think it's good rule of thumb.

I agree with your main argument, of course.

Re:“Let’s use Kubernetes.” Now you have eight problems by geggam on Thursday March 05, 2020 @12:10PM (#22495543)
The overhead managing a container ecosystem to run production is not trivial. IF you are doing this in a service then by all means leverage that package methodology.

If you are managing your systems who already have a robust package management layer then adding the container stacks on top of managing the OS layers you have just doubled the systems your operations team is managing.

Containers also bring NAT and all sorts of DNS / DHCP issues that require extremely senior well rounded guys to manage.

Developers dont see this complexity and think containers are great.

Effectively containers moves the complexity of managing source code into infrastructure where you have to manage that complexity.

The tools to manage source code are mature. The tools to manage complex infrastructure are not mature and the people with the skills required to do so ... are rare.

Re:“Let’s use Kubernetes.” Now you have eight problems by Legogris on Thursday March 05, 2020 @12:24PM (#22495703)
> If you are managing your systems who already have a robust package management layer then adding the container stacks on top of managing the OS layers you have just doubled the systems your operations team is managing.

Oh yeah, if you're not building the software in-house it's a lot less clear that "Containerizate Everything!" is the answer every time. Though there are stable helm charts for a lot of the commonly used software out there, do whatever works for you, man ;)

> Containers also bring NAT and all sorts of DNS / DHCP issues that require extremely senior well rounded guys to manage.

I mean, at that point you can just run with host mode networking and it's all the same, no?

Re:“Let’s use Kubernetes.” Now you have eight problems by scarface74 on Thursday March 05, 2020 @02:31PM (#22497173)
Or you can just use ECS/Fargate and each container registers itself to Route53 and you can just use DNS...
Re:“Let’s use Kubernetes.” Now you have eight problems by scarface74 on Thursday March 05, 2020 @02:26PM (#22497119)
Out of all of the business concerns that a startup or basically any company has, “cloud lock in” is at the bottom of the list.
Re:“Let’s use Kubernetes.” Now you have eight problems by scarface74 on Thursday March 05, 2020 @02:30PM (#22497166)
Creating the infrastructure is using your infrastructure as code framework of choice - Terraform or CloudFormation.

Monitoring can be done with whatever your cloud platform provides.

Re:“Let’s use Kubernetes.” Now you have eight problems by geggam on Thursday March 05, 2020 @10:00AM (#22494014)
Hate to tell you this but that second idea is much easier to manage from an operations standpoint and significantly more reliable.

Also easier to debug and monitor... but you run your business to make developers happy right ?

Re:“Let’s use Kubernetes.” Now you have eight problems by p_l on Thursday March 05, 2020 @11:51AM (#22495311)
Having worked with ECS 2.0 when Fargate was released, the setup was much harder to use from operations standpoint the moment you needed anything more complex (not to mention any kind of state) than k8s. Just getting monitoring out of AWS was annoyance involving writing lambda functions to ship logs...
Re:“Let’s use Kubernetes.” Now you have eight problems by scarface74 on Thursday March 05, 2020 @02:27PM (#22497133)
Fargate has built in logging and monitoring via CloudWatch. It redirects console output to CloudWatch logs.
Re:“Let’s use Kubernetes.” Now you have eight problems by p_l on Thursday March 05, 2020 @05:56PM (#22499260)
And I'll be blunt that CloudWatch, especially logs, aren't all that hot in actual use. It might have gotten better, but all places I worked for last two years if they used AWS they depended on their own log aggregation, even if using AWS-provided Elastic Search.
Re:“Let’s use Kubernetes.” Now you have eight problems by scarface74 on Thursday March 05, 2020 @07:57PM (#22500129)
Re:“Let’s use Kubernetes.” Now you have eight problems by rclayton on Thursday March 05, 2020 @08:54PM (#22500385)
CloudWatch sucks. Sorry.
Re:“Let’s use Kubernetes.” Now you have eight problems by takeda on Thursday March 05, 2020 @10:53AM (#22494592)
I think that was his point (you are referring to using VMs, right?)
Re:“Let’s use Kubernetes.” Now you have eight problems by hedora on Thursday March 05, 2020 @10:27AM (#22494296)
> It's an afternoon worth of research to understand the basic concepts. Then, with the powerful and intuitive tooling you can spin up your own cluster on your computer in minutes and practice deploying containers that.

Source? I’ve never heard of someone going from “what’s kubernetes?” to a bare metal deployment in 4 hours.

Re:“Let’s use Kubernetes.” Now you have eight problems by p_l on Thursday March 05, 2020 @11:49AM (#22495274)
That's more for minikube, which is a few minutes to run, as it is specifically for developer to test things locally on their computer.

The basic concepts in k8s are also pretty easy to learn, provided you go from the foundations up -- I have a bad feeling a lot of people go the opposite way.

Re:“Let’s use Kubernetes.” Now you have eight problems by akvadrako on Thursday March 05, 2020 @03:43AM (#22491960)
That’s a pretty small API. I don’t know what developers are doing if they can’t learn that in stride.
Re:“Let’s use Kubernetes.” Now you have eight problems by rclayton on Thursday March 05, 2020 @09:41AM (#22493850)
Really? If someone told me they were going to write all the glue code that basically gets you the same thing a UI deployment of k8s and a couple yaml files can provide, I’d walk out.
Re:“Let’s use Kubernetes.” Now you have eight problems by p_l on Thursday March 05, 2020 @11:53AM (#22495333)
Reminds me of the business trying to reduce our spend on VMware licenses while keeping the same architecture.

A high level person actually asked me to reimplement vCenter :|