It's worth noting that the Docker experience is very different across platforms. If you just run Docker on Linux, it's basically no different than just running any other binary on the machine. On macOS and Windows, you have the overhead of a VM and its RAM to contend with at minimum, but in many cases you also have to deal with sending files over the wire or worse, mounting filesystems across the two OSes, dealing with all of the incongruities of their filesystem and VFS layers and the limitations of taking syscalls and making them go over serialized I/O.

Honestly, Docker, Inc. has put entirely too much work into making it decent. It's probably about as good as it can be without improvements in the operating systems that it runs on.

I think this is unfortunate because a lot of the downsides of "Docker" locally are actually just the downsides of running a VM. (BTW, in case it's not apparent, this is the same with WSL2: WSL2 is a pretty good implementation of the Linux-in-a-VM thing, but it's still just that. Managing memory usage is, in particular, a sore spot for WSL2.)

(Obviously, it's not exactly like running binaries directly, due to the many different namespacing and security APIs docker uses to isolate the container from the host system, but it's not meaningfully different. You can also turn these things off at will, too.)

Though also, the added complexity to my workflow is an order of magnitude more important to me than the memory usage/background overhead (especially now that we've got the M1 macs, which don't automatically spin up their fans to handle that background VM)

Therefore, if you use Docker 'idiomatically', configuring with environment variables, communicating over the network, and possibly using volumes for the filesystem, it doesn't make your actual backend code any less portable. If you want to doubly ensure this, don't actually tether your build/CI directly to Docker: You can always use a standard-ish shell script or another build system for the actual build process instead.

To that end- I prefer to just stick with modern languages whose first-party tooling makes them not really have to care what OS they're building/running on. That way you can work with the code and run it directly pretty much anywhere, and then if you reserve Dockerfiles for deployment (like in the OP), it'll always end up on a Linux box anyway so I wouldn't worry too much about it being Linux-specific

Agree that it's a strong argument for using newer languages with good tooling and dependency management.

You can just bind the dependencies that are running in Docker to local ports and run your app locally against them without having to docker build the app you're actually working on.

The closest I've come in years to having to really wrestle with it was the URL hacking needed to find the latest version willing to run on my 10-year-old MBP that I expect to run badly on anything newer than 10.13 - the installer was there on the website, just not linked I guess because they don't want the support requests. Once I actually found and installed it, it's been fine, except that it still prompts me to update and (like every Docker Desktop install) bugs me excessively for NPS scores.

It made sense at one point to use Macs but these days pretty much everything is electron or web based or has a Linux native binary. IMHO backend developers should use x64 linux. That's what your code is running on and using something different locally is just inviting problems.

I've used Ubuntu, WSL2 and currently a M1 mac and if I need to be mobile AT ALL with the machine I chose a Mac any day. For a desktop computer Ubuntu works great though

That’s quite the assumption. Graviton is very popular. I haven’t touched x64 stuff in a very long time. Perhaps such generalization is a bad idea.

It may be less than ideally efficient in processor time to have everything I work on that uses Postgres talk to its own Postgres instance running in its own container, but it'd be a lot more inefficient in my time to install and administer a pet Postgres instance on each of my development machines - especially since whatever I'm building will ultimately run in Docker or k8s anyway, so it's not as if handcrafting all my devenvs 2003-style is going to save me any effort in the end, anyway.

I'll close by saying here what I always say in these kinds of discussions: I've known lots of devs, myself included, who have felt and expressed some trepidation over learning how to work comfortably with containers. The next I meet who expresses regret over having done so will be the first.

Maybe you've dealt with Python deployments and been bitten by edge cases where either PyPI packages or the interpreter itself just didn't quite match the dev environment, or even other parts of the production environment. But still, it "mostly" works.

Maybe you've dealt with provisioning servers using something like Ansible or SaltStack, so that your setup is reproducible, and run into issues where you need to delete and recreate servers, or your configuration stops working correctly even though you didn't change anything.

The thing that all of those cases have in common is that the Docker ecosystem offers pretty comprehensive solutions for each of them. Like, for running containers, you have PaaS offerings like Cloud Run and Fly.io, you have managed services like GKE, EKS, and so forth, you have platforms like Kubernetes, or you can use Podman and make a Systemd unit to run your container on a stock-ish Linux distro anywhere you want.

Packaging your app is basically like writing a CI script that builds and installs your app. So you can basically take whatever it is you do to do that and plop it in a Dockerfile. Doesn't matter if it's Perl or Ruby or Python or Go or C++ or Erlang, it's all basically the same.

Once you have an OCI image of your app, you can run it like any other application in an OCI image. One line of code. You can deploy it to any of the above PaaS platforms, or your own Kubernetes cluster, or any Linux system with Podman and Systemd. Images themselves are immutable, containers are isolated from eachother, and resources (like exposed ports, CPU or RAM, filesystem mounts, etc.) are granted explicitly.

Because the part that matters for you is in the OCI image, the world around it can be standard-issue. I can run containers within Synology DSM for example, to use Jellyfin on my NAS for movies and TV shows, or I can run PostgreSQL on my Raspberry Pi, or a Ghost blog on a Digital Ocean VPS, in much the same motion. All of those things are one command each.

If all you needed was the static binary and some init service to keep it running on a single machine, then yeah. Docker is unnecessary effort. But in most cases, the problem is that your applications aren't simple, and your environments aren't homogenous. OCI images are extremely powerful for this case. This is exactly why people want to use it for development as well: sure, the experience IS variable across operating systems, but what doesn't change is that you can count on an OCI image running the same basically anywhere you run it. And when you want to run the same program across 100 machines, or potentially more, the last thing you want to deal with is unknown unknowns.

If you freelance and work on different project, sure rvm is a great thing, but docker will contain it even better and you won't litter your work machine with stuff like mine is after a few years.

I don't totally understand the maintenance story. On heroku with buildpacks, I don't need to worry about OS-level security patches, patches to anything that was included in the base stack provided by the PaaS, they are responsible for. Which I consider part of the value proposition.

When I switch to using "Docker" containers to specify my environment to the PaaS... if there is a security patch to the OS that I loaded in one of my layers... it's up to me to notice and update my container? Or what? How is this actually handled in practice?

(And I agree with someone else in this thread that this fly.io series of essays on docker is _great_, it's helping me understand docker better whether or not I use fly.io!)

But it sounds like for patch/update management purposes, now I need to add something like that in? Another platform/host to maintain/manage, at possibly additional price, and then we add dealing with the specific mechanisms for scanning/updating too...

Bah. It remains mystifying to me that the current PaaS docker-based "best practices" involve _quite a bit_ more management than heroku. I pay for a PaaS hoping to not do this management! It seems odd to me that the market does not any longer seem to be about providing this service.

It is odd, but the general direction of the market (at least at the top) is becoming less opinionated, which means you need to bring your own. Not sure I like that myself.

Re: security updates. It's not handled. There are companies that will scan your infrastructure to figure out what's in your containers and find out of date OS base images.

One thing I'm currently experimenting with is a product for people who would like an experience slightly closer to traditional Linux. The gist is, e.g.

    include "#!gradle -q printConveyorConfig"   // Import Java server
    app {
      deploy.to = "vm-frontend-{lhr,nyc}{1-10}.somecloud.com"
      linux { 
         services.server {
            include "/stdlib/linux/service.conf"
         }
         debian.control.Depends = "postgres (>= 14)"
      }
    }

SystemD can also run containers and "portable services", as well as using cgroups for isolation and sandboxing, so there's no need to use debs specifically. It just means that you can depend on stuff that will get upgraded as part of whole OS system upgrades.

We use this to maintain our own servers and it's quite nice. One of the things that always bugged me about Linux is that deploying servers to it always looks like either "sftp a tarball and then wire things up yourself using vim", or Docker which involves registries and daemons and isolated OS images, but there's nothing in between.

Not sure whether it's worth releasing though. Docker seems so dominant.

So... what do actual real people do in practice? I am very confused what people are actually doing here.

LOTS of people seem to have moved to this kind of docker-based deploy for PaaS. They can't all just be... ignoring security patches?

I am very confused that nobody else seems to think the "handle patches story" is a big barrier to moving from heroku-style to docker-style... that everyone else just moves to docker-style? What are they actually doing to deal with patches?

I admit I don't really understand your solution -- I am not a sysadmin! This is why we deploy to heroku and pay them to take care of it! It is confusing to me that none of the other PaaS competitors to heroku -- including fly.io -- seem to think this is something their customers might want...

The Dockerfile has in it:

> ARG RUBY_VERSION=3.2.0

> FROM ruby:$RUBY_VERSION

Which it says gets us "gets us a Linux distribution running Ruby 3.2." (Ubuntu?). The version of that "linux distribution" isn't mentioned in the Dockerfile. How do I "bump" it if there is a security patch effecting it?

Then it has:

> RUN apt-get update -qq && apt-get install -y build-essential libvips && \ ...

Then I just run `fly launch` to launch that Dockerfile on fly.io. How do I "bump" the versions of those apt-get dependencies should they need them?

If it's not, you just need to trigger a build every so often. Maybe this could be a feature PaaS offers in the future.

There are a few problems with this, as noted elsewhere in the thread:

1. You have to do an uncached rebuild and repush all of your images, using some ad-hoc company specific process. There's nothing that can do this for you at the end points or service levels, because Docker images are meant to be immutable after build and don't come with the scripts or inputs used to build them.

2. The default is to use caching, so devs may not notice that they didn't refresh their base OS for a while.

3. You don't get notified when updates are available or applied. There's nothing like the unattended-upgrades package that comes with Debian normally, which will apply upgrades and then tell you what happened.

4. Because of (3) the latency is very high. There is no story (other than third party scanners) for getting notified about urgent upgrades. If there's another zero day in OpenSSL then with a standard Linux install you'll get patched as soon as a new package is released and your machines update, so pretty quick (a day or so). With Docker images, it'll get patched on an app-by-app basis if and when people get around to doing an uncached rebuild and repush of the image.

5. Kernel upgrades are a whole can of eels in container-world. People like to think of the container as being a self-contained OS but it isn't. There is a largely unstated and untested assumption that any Linux distro user space can run on any kernel version or configuration, regardless of whether the OS originally shipped in that configuration, and everything will just automatically do something sensible. Mostly this assumption is OK because servers are very simple, but it's not actually guaranteed by anything. A lot of people misunderstand the "stable Linux syscall interface" guarantees and what that means.

It's for reasons like this that I prefer the slightly older way of running real binaries that are exposed to the OS and which use OS specific packages. I configured unattended-upgrades and use LTS versions of the OS, so that security patches just stream in without me doing anything. There are a few downsides to this too:

1. You have to remember to restart your servers from time to time, or schedule it to happen on a regular basis. The OS will patch stuff on disk, but not in memory. The Debian security patching story is thus not really complete. Of course this is still better than with Docker!

2. You do need to understand at least a bit of Linux sysadmin. Enough to know how to ssh in as root, use apt-get and so on.

3. The tooling story is poor, hence my musings above about demand for something better. Without Docker, today you're going to be manually copying files to the server, having to learn systemd and how to start/stop/enable services, how to restart them on upgrades etc. That's why I've written something that does it all for you.

They can and they are, IME. I've seen images that literally run some version of Alpine Linux from 3 years ago.

Right. I didn't explain it very well then.

Basically it's a heroku-ish solution but as a tool rather than a service, and where you build locally instead of pushing source code to some remote cloud. You say "here's my build system, go push to these plain Linux VMs". Now, someone needs at least a bit of sysadmin knowledge - you need to know how to obtain Linux VMs, set up access to them, and make sure they're (self-)applying security updates. From time to time you'll need to roll to new OS releases and do restarts for kernel fixes. But that stuff isn't all that hard to learn.

Still - I'd be curious to know where your threshold is for touching Linux. With Heroku you never did, right? Dynos could have run Windows for all you knew? If you had a tool that e.g. you gave your cloud credentials to, and it then spun up N Linux VMs, logged in, configured automatic updates and some basic monitoring then let you push servers straight from your git repo, would you be interested in that? How much did you rely on Heroku support going in and helping you fix app-specific problems live in production?

I wouldn't be surprised if I'm wrong and newer tools bridge the gap for a lower price and/or time investment, but I also wouldn't be surprised if I'm right and many places using Docker could save time/money offloading the maintenance to something more like a managed PaaS.

There's also netlify, vercel and similar sites but I think they're mainly geared toward all-javascript apps.

If you're really concerned, just have a CI job that rebuilds and tests with newer base image versions.

If someone updates nodejs, just pull in the latest dockerfile. Someone updates to a new postgres version? Same thing. It's so much better than managing native dependencies.

Have you ever worked somewhere where in order to run the code locally on your machine, it's a blend of "install RabbitMQ on your machine, connect to MS-SQL in dev, use a local Redis, connect to Cassandra in dev, run 1 service that this service routes through/calls to locally, but then that service will call to 2 services in dev", etc?

IMO Docker for local dev is most beneficial for python where local installs are so all over the place.

If anyone is curious about the details, I simply reused the existing `bin/dev` script set up by Rails by adding this to `Procfile.dev`:

    docker: docker compose -f docker-compose.dev.yml up

For your issue could you handle bringing up your docker-compose 'manually' in bin/dev? Maybe conditionally by checking if the image exists locally with `docker images`. Then tear it down and run foreman after it completes?

And 50% of the time it actually works. The other 50% of the time, it doesn't, and I spend a week talking to other engineers and ops trying to figure out why.

Not a jab at Docker, btw.

This way each developer can/should tear down and rebuild their entire development environment on a semi-frequent basis. That way you know it works way more than 50% of the time. (Stretch goal: put this in CI.... :)

The other side benefit: if you reset/restore your development database frequently, you are incentived/forced to add any necessary dev/test data into the "rake generate_development_data", which benefits all team members. (I have thought about a "generate_development_data.local.rb" approach where each dev could extend the generation if they shouldn't commit those upstream, but haven't done that by anymeans....)

That said, I recently discovered that VS Code has a feature called "Dev Containers"[0] that ostensibly makes it easy to develop inside the same Docker container you'll be deploying. I haven't had a chance to check it out, but it seems very cool.

[0] https://code.visualstudio.com/docs/devcontainers/containers

    web:
      image: rubylang/ruby:3.0.1-focal
      volumes:
        - .:/myapp

Currently, I'm working in 3 projects at once + my own. Even though it's all PHP, the projects are PHP 8.1 / Symfony 6.2, PHP 8.2 / Symfony 6.2, PHP 8.1 / Laravel 8 and the newest project I joined is PHP 7.4 / Symfony 4. I have to adhere to different standards and different setups; I don't want to switch the language and the yarn or npm versions or the Postgres or MySQL versions and remember each one. Some might use RabbitMQ, another Elasticsearch.

Docker helps me tremendously here.

Additionally, I add a Makefile for each project as well and include at least "make up" (to start the whole app), "make down", "make enter" (to enter the main container I'll be working with), "make test" (usually phpunit) and perhaps "make prepare" (for phpunit, phpstan, php-cs-fixer, database validate) as a final check before adding a new commit.

Locally, I had additional aliases on my shell. "t" for "make test", "up" for "make up".

So now I just have to go to a project folder, write "up" and am usually good to go!

Modern languages with modern tooling can automatically take care of dependency management, running your code cross-platform, etc [0]. And several of them even have an upgrade model where you should never need an older version of the compiler/runtime; you just keep updating to the latest and everything will work. I find that when circumstances allow me to use one of these languages/toolsets, most of the reasons for using Docker just disappear.

[0] Examples include Rust/cargo, Go, Deno, Node (to an extent)

If the new project without any tests with a PHP 7.2 and MySQL installation and I need to upgrade it to PHP 8.2, I first need to write tests and I can't use PHP 8.2 features until I've upgraded.

Composer is the dependency manager, but I still need PHP to run the app later on. And a PHP 7.2 project might behave differently when running on PHP 7.2 or PHP 8.2. And sometimes PHP is just one part of the equation. There might be an Angular frontend and a nginx / php-fpm backend, maybe Redis, Postgres, logging, etc. They need to be wired together as well. And I'm into backend web development and do a bit of frontend development, but whoa, I get confused with all the node.js, npm, yarn versioning with corepack and nvm and whatnot. Here even a "yarn install" behaves differently depending on which yarn version I have. I'd rather have docker take care of that one for me.

I feel like "docker (compose)" and "make" are widely available and language-agnostic enough and great for my use cases (small to medium sized web apps), especially since I develop on Linux.

Something language-specific like pyenv might work as well, but might be too lightweight for wiring other tools. I used to work a lot with Vagrant, but that seems to more on the "heavy" side.

Edit: I just saw your examples, unfortunately I've only dabbled a bit with Go and haven't worked with Rust yet, so I can't comment on that, but would be interested to know, how they work re: local dev setup.

Yeah- so in Rust, the compiler/tooling never introduces breaking changes, as (I think) a rule. For any collection of Rust projects written at different times, you can always upgrade to the very latest version of the compiler and it will compile all of them.

The way they handle (the very rare) breaking changes to the language itself is really clever: instead of a compiler version, you target a Rust "edition", where a new edition is established every three years. And then any version of the Rust compiler can compile all past Rust editions.

Node.js isn't quite as strict with this, though it very rarely gets breaking changes these days (partly because JavaScript itself virtually never gets breaking changes, because you never want to break the web). Golang similarly has a major goal of not introducing breaking changes (again, with some wiggle-room for extreme scenarios).

> I get confused with all the node.js, npm, yarn versioning with corepack and nvm and whatnot. Here even a "yarn install" behaves differently depending on which yarn version I have.

Hmm. I may be biased, but I feel like the Node ecosystem (including yarn) is pretty good about this stuff. Yarn had some major changes to how it works underneath between its major versions, but that stuff is mostly supposed to be transient/implementation-details. I believe it still keys off of the same package.json/yarn.lock files (which are the only things you check in), and it still exposes an equivalent interface to the code that imports dependencies from it.

nvm isn't ideal, though I find I don't usually have to use it because like I said, Node.js rarely gets breaking changes. Mostly I can just keep my system version up to date and be fine, regardless of project

Configuring the Node ecosystem's build tools gets really hairy, but once they're configured I find them to mostly be plug and play in a new checkout or on a new machine (or a deployment); install the latest Node, npm install, npm run build, done. Deno takes it further and mostly eliminates even those steps (and I really hope Deno overtakes Node for this and other reasons).

> maybe Redis, Postgres, logging, etc. They need to be wired together as well

I think this - grabbing stock pieces off the shelf - is the main place where Docker feels okay to use in a local environment. No building dev images, minimal state/configuration. Just "give me X". I'd still prefer to just run those servers directly if I can (or even better, point the code I'm working on at a live testing/staging environment), but I can see scenarios where that wouldn't be feasible

What if you want to start a new project using the latest postgres version because postgres has a new feature that will be handy, but you already maintain another project that uses a postgres feature or relies on behaviour that was removed/changed in the latest version? You're going to set up a whole new VM on the internet to be a staging environment and instead of setting up a testing and deployment pipeline you're going to just FTP / remote-ssh into it and change live code?

you define an apps entire chain of dependencies including external services in a compose file / set of kube manifests / terraform config for ecs. Then in the container definition itself you lock down things like C library and distro versions: maybe you use specially patched imagemagick on one project or a pdf generator on another, and fontconfig defaults were updated and it changed how aliasing works between distro releases and now your fonts are all fugly in generated exports... stick all those definitions in a Dockerfile and deploy onto any Linux distro / kernel and it'll look identical to it does on local

nevermind this, check out this thread to destroy your illusion that simply having node installed locally will make your next project super future proof https://github.com/webpack/webpack/issues/14532 and note that some of the packages referencing this old issue in open new bug reports are very popular!

if you respond please do not open with "yeah but rust", I can still compile Fortran code too

I have to assume you work by yourself or in an extremely small company to be able to handle project complexity without docker.