News for hackers. Stuff that matters.
Re:Bare Metal K8s Clustering at Scale by reacharavindh on Tuesday July 03, 2018 @03:48AM (#17448292)
Wish the article had more context for readers.

These days it feels like everybody needs to throw in Kubernetes at everything introducing complexity for the sake of being cool.

I guess those of us that likes to run non-distributed software for small scale applications are the new grumpy grey beards....

Re:Bare Metal K8s Clustering at Scale by eddieroger on Tuesday July 03, 2018 @08:11AM (#17449472)
Maybe it's because I'm also in retail IT and have seen similar models, but I get it. At the risk of oversimplifying, they've got two problems - code written in an office somewhere has to run identically in 2000 geographically separated places, and it needs to do so with some level of protection against failure. All other K8s gloss aside, it's a good way of making sure that there are multiple instances of a Docker container running. Assuming they set the cluster up correctly, they can suffer hardware failure and not downtime, and assuming they're using those little nucs in the photo, they can do so for under $1500. Sure a single server-grade piece of hardware would probably do just fine and have similar protection from failure, but this is essentially that on commodity hardware.
Re:Bare Metal K8s Clustering at Scale by coffeesn0b on Tuesday July 03, 2018 @01:19PM (#17451977)
Our cost is actually something like $900 for the whole cluster :) . Consumer grade clustering FTW!
Re:Bare Metal K8s Clustering at Scale by eddieroger on Wednesday July 04, 2018 @10:54AM (#17457576)
That's awesome. Nice article, by the way. How much of the restaurant runs on these clusters, if you can share (rough percentages are fine). I doubt there's a whole POS server on there, but maybe?
Re:Bare Metal K8s Clustering at Scale by coffeesn0b on Wednesday July 04, 2018 @10:59PM (#17460583)
We're still in the process of rolling things out, although I'm not sure I can give a percentage... I think it's safe to say that in the next 6-12 months most (if not all) of them should have an active cluster.

And no, there is not a full POS server on them at this time... it will take us a few years to decompose that monolith, but this is a natural place to put it as we do so.

Re:Bare Metal K8s Clustering at Scale by chx on Tuesday July 03, 2018 @07:45AM (#17449335)
> for small scale applications

I am a grumpy grey beard no doubt but I still maintain: most websites do not need more than a single server -- certainly not more than a single database server. And, for most, a few hundred dollar dedidcated server is aplenty. Apply YAGNI until blue in the face.

Re:Bare Metal K8s Clustering at Scale by StavrosK on Tuesday July 03, 2018 @08:29AM (#17449574)
I'm constantly surprised at how much deployments cost. AWS makes it easy to spin up servers, so people do, and they have tens of machines sitting idle when one would do. It seems that people saw containers and thought "finally, each instance can easily have its own machine", whereas I've always thought of containers as tools for putting many things on the same machine.

I run a Dokku instance on a Hetzner server and it's been fantastic, I host 10-20 of my projects with thousands of daily users there without it even breaking a sweat. My only regret is that I should have used a single database instead of one DB per project, but I was lazy.

Re:Bare Metal K8s Clustering at Scale by bitwize on Tuesday July 03, 2018 @03:00PM (#17452802)
Sorry old timer, servers are cattle now. Actually more like Shmoos: so abundant as to be almost free. And if a server is 90% reliable, each additional server adds an additional 9 of reliability. So it always makes sense to plan for more than one, or even two, servers.
Re:Bare Metal K8s Clustering at Scale by dvanduzer on Tuesday July 03, 2018 @11:27AM (#17450952)
A five minute outage of the point-of-sale system during the lunch rush can easily cost even the smallest of restaurants several hundred dollars.

True, most websites do not have this problem, because most websites do not drive revenue like that. There are plenty of use cases where you need five nines, but only within limited not-24/7 time windows.

Re:Bare Metal K8s Clustering at Scale by chx on Tuesday July 03, 2018 @12:51PM (#17451716)
Two servers... still doesn't need Kubernetes.
Re:Bare Metal K8s Clustering at Scale by coffeesn0b on Tuesday July 03, 2018 @01:18PM (#17451969)
It's actually 3 nodes, with plans to expand out to more based on workload.
Re:Bare Metal K8s Clustering at Scale by itchyouch on Tuesday July 03, 2018 @08:21AM (#17449525)
In my particular environment, we have over 15k jvms running across 2k hosts just for our US applications. K8s absolutely makes sense.

But for non-distributed software with only several clients, the traditional model is still fine. E.g. we still run gitlab as a pet to serve our cattle infrastructure.

Re:Bare Metal K8s Clustering at Scale by colonelpopcorn on Tuesday July 03, 2018 @09:46AM (#17450118)
Do you have scripts that could rebuild your pet Gitlab server? Beyond the storage requirements and stateful components, getting Gitlab to bootstrap from a base image should be pretty trivial, right?
Re:Bare Metal K8s Clustering at Scale by throwbacktictac on Tuesday July 03, 2018 @04:15AM (#17448384)
I was curious about what problem was being solved too. I can't imagine what a chicken restaurant needs with a distributed k8's cluster.
Re:Bare Metal K8s Clustering at Scale by organsnyder on Tuesday July 03, 2018 @08:08AM (#17449452)
I'm wondering the same thing, and it looks like a lot of other people are too—from the bottom of the article:

> Edit 7/2/18 — since writing this, many readers have asked “why not just use the cloud? Why computing at the Edge?”. We have realized that we did not provide much context about why we’re doing what we’re doing, so we will follow up with an post about that soon.

Re:Bare Metal K8s Clustering at Scale by shaklee3 on Tuesday July 03, 2018 @11:42AM (#17451091)
They somewhat cleared it up in a comment. Low latency was one reason.
Re:Bare Metal K8s Clustering at Scale by coffeesn0b on Tuesday July 03, 2018 @01:16PM (#17451951)
(I'm re-pasting this intro to a few posters)

Hey! I'm Caleb, the SRE that helped build this solution...

Sorry about the lack of context in the article, it was intended for a specific audience (QCon) where we gave a lot more context to the problem at hand.

What we were trying to solve for was; 1) Low latency 2) High Availability 3) Container based, zero-downtime deployments 4) Continued operations even in an internet-down event

If you're running applications in a few locations, in small scale environments, our approach would be way overkill for that problem set.