News for hackers. Stuff that matters.
Re:Swarm vs. Fleet vs. Kubernetes vs. Mesos by KirinDave on Friday October 23, 2015 @12:41PM (#10439704)
Only Kubernetes has facilities for hosting external services reliably. I'm not sure why these other lower level tools are being compared to it. Especially Mesos and Marathon, which are much lower level.
Re:Swarm vs. Fleet vs. Kubernetes vs. Mesos by amouat on Friday October 23, 2015 @01:27PM (#10439993)
They are being compared as they are the main options for clustering and scheduling containers. I'd agree Kubernetes is at a higher level to the other options, or at least comes with more features.

I'm not sure what you mean by "hosting external services reliably" - what's external and who is unreliable?

Re:Swarm vs. Fleet vs. Kubernetes vs. Mesos by amouat on Friday October 23, 2015 @01:33PM (#10440032)
Hmm, I'll answer my own question - you mean has a load-balancer built in, don't you?
Re:Swarm vs. Fleet vs. Kubernetes vs. Mesos by KirinDave on Friday October 23, 2015 @01:36PM (#10440051)
And that doesn't recommend a comically brittle solution (as in marathon) or provide no facilities at all (as in swarm).

There's nothing wrong with these decisions. It's just odd to compare this group. It makes me wonder how people are considering these options and in what context.

You cannot field a reliable and available service to the outside world without this component. And the instrumentation and operationalization of this component is tricky.

Re:Swarm vs. Fleet vs. Kubernetes vs. Mesos by amouat on Friday October 23, 2015 @01:49PM (#10440141)
Yeah, I agree that the load-balancer is important and not adequately addressed. I guess most people rig up their own set-up based on haproxy currently.

I don't get why you think I can't compare that group though. What should I compare Kubernetes with? The group represents the various approaches to clustering and orchestration that most people will consider for container deployments. They are all slightly different, which is what makes for an interesting comparison.

Re:Swarm vs. Fleet vs. Kubernetes vs. Mesos by KirinDave on Friday October 23, 2015 @02:00PM (#10440217)
K8S pretty much stands on its own in the Docker ecosystem right now. I'm not aware of good alternatives that aren't comically janky as you described.
Re:Swarm vs. Fleet vs. Kubernetes vs. Mesos by manojlds on Friday October 23, 2015 @01:10PM (#10439887)
Marathon and Kubernetes are in the same level, why do you call it lower level. Mesos, yes, and Kubernetes can run on Mesos too.
Re:Swarm vs. Fleet vs. Kubernetes vs. Mesos by KirinDave on Friday October 23, 2015 @01:18PM (#10439940)
Maraton schedules long running tasks, but doesn't offer any explicit facilities for binding a homogenous group into a relible responder set.

At least, last time I checked. I can see components in the Marathon API to help you build such a thing. But Kubernetes deals with a fundamentally more abstract set of primitives: groups of containers (pods), replicated pods, and automatically managed dispatch points to said groups called services.

The Marathon documentation literally recommends you hand-roll a periodically updated haproxy service to accomplish that last bit. While K8S introduces a fair amount of overhead to accomplish this feat reliably, that overhead pays off by having a very responsive and available system.

So yeah. K8s is working one level up the stack, as I see it. I think you can make something reasonable with only Marathon, and that thought is backed up by the reality of people doing it. But my own experience suggests K8S does more for you here. Hence my excitement about K8S-mesos.

Re:Swarm vs. Fleet vs. Kubernetes vs. Mesos by ersoft on Friday October 23, 2015 @03:07PM (#10440659)
I managed to achieve load balancing for a fleet of microservices using vulcand [0] and marathon callbacks [1].

It listens for marathon callbacks (for app create/update/scale), or when a task is killed/lost and updates the backends in etcd, propagating to all vulcand load balancers.

The tool is available on github [2]

[0] https://vulcand.io/proxy.html#backends-and-servers

[1] https://mesosphere.github.io/marathon/docs/event-bus.html

[2] https://github.com/kuende/kameni

Re:Swarm vs. Fleet vs. Kubernetes vs. Mesos by KirinDave on Saturday October 24, 2015 @02:50AM (#10442721)
This is a good thing. Perhaps you could get the Maraton folks to stop recommending such an awkward solution in preference to your work?
Re:Swarm vs. Fleet vs. Kubernetes vs. Mesos by ersoft on Saturday October 24, 2015 @06:18AM (#10443110)
Well, it depends. My solution is opinionated, using only one vulcand cluster for load balancing, multiple vulcand servers listening on one etcd namespace.

For a small number of nodes (< 100) and requests (< 8000 req/s on each vulcand server) this vulcand approach is ok. For large scale, HAProxy/Nginx supports a lot more req/s than vulcand, and I think it can also be configured using confd [0] using a similar aproach: listen for marathon events, update etcd keys, then confd will listen for changes and reload HAProxy/Nginx

[0] http://ox86.tumblr.com/post/90554410668/easy-scaling-with-do...

Re:Swarm vs. Fleet vs. Kubernetes vs. Mesos by KirinDave on Friday October 23, 2015 @01:23PM (#10439973)
Just to double down on that previous point, check out the code for haproxy-marathon-bridge.

https://github.com/mesosphere/marathon/blob/master/bin/hapro...