News for hackers. Stuff that matters.
Re:Billions wasted on Hadoop startups, the same will eventua... by pixelrevision on Friday May 31, 2019 @09:26AM (#20060621)
It’s very easy to underestimate how helpful it can be when you first start working with it. It’s a black box that uses root to do everything and is a pain to debug. Because of this it becomes easy to hand wave it away.

Once you have a properly setup project going and your entire build process is mostly repeatable the benefits start becoming more obvious. Yes, you can do all the same things to a certain extent in a vm but it’s really hard to keep that streamlined and up to date. Having a script that sets up your stack in a vm on both windows and Mac then runs on Linux is a pretty big maintenance nightmare. A dockerfile works with a few commands and can be added to your repo.

It’s not without trade offs but I think if they can solve the issue of debugging in a better way then we’ll really see things solidify on this concept.

Re:Billions wasted on Hadoop startups, the same will eventua... by hardwaresofton on Friday May 31, 2019 @09:37AM (#20060707)
> It’s a black box that uses root to do everything and is a pain to debug.

This is less true these days (and on macs that had to use the docker machine "hack" it was barely ever true, per say) -- rootless containers are on the way thanks to user namespaces. For example LXC can run fully rootless containers that act more like VMs themselves (as in they will have systemd as pid 1 inside) -- kernel support, user namespaces, and File systems in Userspace (FUSE) make this possible.

IMO One of the biggest benefits to running containers is actually E2E tests -- I don't see it done as often as I should, but it has become drastically easier to run an entire postgres instance for a single local E2E test runs -- I do this on almost every project I start now, I set up E2E tests that spin up the actual world I expect in production (so all the backing services, at the version they will run at), and interact with my application -- this is a huge step forward compared to a huge wiki that has "how to set up the local test VM" -- you can spin up and shutdown these services so fast that you can use them ephemerally (I do) without much worry.

Re:Billions wasted on Hadoop startups, the same will eventua... by panpanna on Friday May 31, 2019 @10:25AM (#20061166)
Lxc/lxd is a fantastic piece of software that I wish more people would try. Especially if people already on Ubuntu, Debian and Arch.

You absolutely don't need to be a cloud person to use containers. For example I use lxc on my laptop to keep different projects separate and test new software.

Re:Billions wasted on Hadoop startups, the same will eventua... by hardwaresofton on Friday May 31, 2019 @10:53AM (#20061419)
I agree -- it also offered more isolation (user namespaces) from very early on and is way more featureful (live migration has been a thing in LXC for a while IIRC).
Re:Billions wasted on Hadoop startups, the same will eventua... by xrd on Friday May 31, 2019 @12:14PM (#20062224)
Have any links to share that you really liked as examples? I'm using docker compose to do what I think you are saying you do with just pure lxc. I would love to know more about how you do it.
Re:Billions wasted on Hadoop startups, the same will eventua... by panpanna on Friday May 31, 2019 @12:31PM (#20062407)
Not really. I personally use each container like a full VM with it's own software. I don't try to combine a lot of small containers into a system the way you would do with docker.

Here is a general tutorial in anyone wants to test lxc:

https://linuxcontainers.org/lxd/getting-started-cli/

Re:Billions wasted on Hadoop startups, the same will eventua... by hiram112 on Friday May 31, 2019 @12:17PM (#20062255)
> One of the biggest benefits to running containers is actually E2E tests...

100% agree. Most developers don't even bother testing code that requires DBs and other external services because it's a pain to keep test environments up and running so a lot of the most important and buggy code slips through the cracks.

Then you have the developers who do test but just make heavy use of mocking libs and other tricks which aren't helpful in the long run since the mocks don't actually do anything.

But setting up a docker compose with the whole stack - completely reset each time the tests are run - makes integration tests so much more valuable. And usually it's not a huge deal to set up the plumbing since the docker files are often already created for production.

Re:Billions wasted on Hadoop startups, the same will eventua... by Gravityloss on Friday May 31, 2019 @12:21PM (#20062294)
Docker containers must run on a vm on Windows and Mac.

So it's an innovation in packaging.