News for hackers. Stuff that matters.
Re:Billions wasted on Hadoop startups, the same will eventua... by hardwaresofton on Friday May 31, 2019 @09:29AM (#20060651)
Your point is actually the point I'm trying to make -- you shouldn't be using docker to try and contain possibly malicious code, it's not for that.

It does isolate a process's view of the filesystem, it does isolate a process's view of the PID namespace, and that is valuable. Let's say it's a spectrum:

raw processes -------- processes with a certain user ---------- namespace+cgroup isolated processes ---------------- VMs

Docker is certainly an improvement for processes that you don't want running completely contained but want somewhat isolated. For isolation in every sense, you want a VM (a lightweight one if you can get it, i.e. some stripped down qemu).

> MicroVMs start faster and provide better isolation.

Agreed on isolation -- VMs are hands down better at isolating, but I'm a bit skeptical about starting faster.

> Meanwhile, none of this is relevant unless you're building your own cloud platform, which is a huge waste of time for most companies.

Again, this is exactly why it's relevant, 99% of people aren't building their own cloud platform, so they don't need the full isolation of VMs -- most of the time they're just trying to prevent program A that their devs wrote from clobbering program B that their devs also wrote when they both run on the same machine, and making it easier to deploy the dependencies that come with each.

> MicroVMs, containers, VMs, zones and bare metal are places to execute code. Serverless makes all those distinctions irrelevant.

OK I don't even really know what this means, you know serverless runs on MicroVMs right? and most of the time it's actually containers in MicroVMs? MicroVMs are just stripped down versions of regular VMs, and no one is in a zone unless they're running Solaris.

The distinction between these things is still very important, unless you mean that the future is everyone just deploying functions for their applications? But even if you mean that, cold start is basically the first stumbling block you see and it literally exists because of the distinction between how these technologies work (and how fast they can be started on demand, with how much isolation).

> Sorry if you spent 2015 getting really into Docker. You bet on the wrong horse. It's OK, this happens in tech.

Is there anyone that spent 2015 really getting into Docker and isn't better for it now? The technologies that have sprung out of this part of computing are very valuable to know and are getting more valuable, not less. You don't have to install a VM to run an isolated postgres instance on your dev machine because containerization exists -- if you're still doing this you should probably look into updating your tooling.

Also, don't forget that containerization is how some of the richest and supposedly best (due to their ability to spend money on engineers) companies in the world have been handling deployment for nearly a decade -- 2015 is late for realizing containerization is a good thing, not early.

Re:Billions wasted on Hadoop startups, the same will eventua... by nailer on Friday May 31, 2019 @09:46AM (#20060780)
> you shouldn't be using docker to try and contain possibly malicious code

Indeed. And since we're not building our own cloud environment, because that's a waste of resources for most companies, we will share an environment with possibly malicious code so therefore need isolation.

> unless you mean that the future is everyone just deploying functions for their applications?

Yes, that is exactly what I mean.

> You know serverless runs on MicroVMs right?

Yes, that's why I mentioned them

> and most of the time it's actually containers in MicroVMs?

I doubt this - AWSs performance documentation focuses on MicroVMs as an alternative to containers, not an addition. Which makes sense as containers do less than MicroVMs.

But hey, even if the AWS MicroVM documentation is wrong, it doesn't matter. I am not building a cloud platform. I do not care.

You're right about spin up time. Open a socket and let your apps terminate with it open for a greater chance of reuse. As a FaaS user that's your entire concern with your execution environment.

> 99% of people aren't building their own cloud platform, so they don't need the full isolation of VMs

All cloud environments must provide isolation between customers, hence VMs / MicroVMs. Customers adding docker on top of that add a huge administrative overhead that duplicates the features of their cloud provider for littl benefit,

> Is there anyone that spent 2015 really getting into Docker and isn't better for it now?

Every single person whose product is not Internet infrastructure, that wrote or configured their own unnecessary custom LXC or Docker and VM environment because 'Docker changes everything'.

If a company's product is machine learning for detecting cancer and their ops person has a custom Docker/kubernetes environment they're misusing their employer for their own technical interest

>2015 is late for realizing containerization is a good thing, not early.

It is indeed. It's just that then Docker hype was at its maximum.

Re:Billions wasted on Hadoop startups, the same will eventua... by hardwaresofton on Friday May 31, 2019 @10:08AM (#20060997)
> Yes, that is exactly what I mean.

OK, I'd like to note that it's also the past -- CGI was (and in some dark corners still is) a thing.

> Every single person whose product is not Internet infrastructure, that wrote or configured their own unnecessary custom LXC or Docker and VM environment because 'Docker changes everything'.

Yeah but those people now have way easier to run local environments?

> If a company's product is machine learning for detecting cancer and their ops person has a custom Docker/kubernetes environment they're misusing their employer for their own technical interest

???? If your company's product is machine learning, and developers who must work on that product need to set up their environment on their local machines, docker is easier to get started with than VMs, runs faster, consumes less resources. While it might be arguable that it's easier, there are literal money savings to be had by running a docker container instead of a full VM.

Kubernetes has much more complexity and many more tradeoffs involved so I can see that being a much heavier decision.

What you're saying is that overzealous ops people who are looking to pad their own resumes should not be allowed to run amuck, and I agree with that, but docker is not the poster boy for engineering largess -- and I'd argue it never was. Companies and research groups/smaller distributions have been using containerization very productively for a long time.

Re:Billions wasted on Hadoop startups, the same will eventua... by nailer on Friday May 31, 2019 @10:30AM (#20061217)
> CGI was (and in some dark corners still is) a thing.

Yep. Tech does that - think about centralisation / distribution every few years, maybe it'll cycle back to people caring about their own containment tech in future. But not right now.

> Yeah but those people now have way easier to run local environments?

OK, so they wasted their time on their awful custom Docker/k8s thing that runs on top of EC2 anyway, and they have a slightly better way to spin up dev environments?

The rest of the conversation is about dev environments, bare metal, VMs and containers all have their place and I mostly agree with you (obviously containers are only useful when there's a Linux kernel on that desktop, Windows and Mac are bare metal or virtualising for the most part).

> What you're saying is that overzealous ops people who are looking to pad their own resumes should not be allowed to run amuck, and I agree with that, but docker is not the poster boy for engineering largess -- and I'd argue it never was.

You understand my point perfectly. I believe that docker is precisely the poster boy for engineering largess, but this is based on my own experiences (talking to a lot of young engineers in the startup world who love wasting investor money on ops) and it seems reasonable that you have had different experiences.

I think we have a good understanding of where we each come from and can end it here. Thanks for being civil.