Rails on Docker · Fly

Re:Rails on Docker · Fly by mike_hearn on Thursday January 26, 2023 @01:21PM (#34536145)

Yeah the fly.io stuff is great, beautiful artwork too, top class job.

Re: security updates. It's not handled. There are companies that will scan your infrastructure to figure out what's in your containers and find out of date OS base images.

One thing I'm currently experimenting with is a product for people who would like an experience slightly closer to traditional Linux. The gist is, e.g.

    include "#!gradle -q printConveyorConfig"   // Import Java server
    app {
      deploy.to = "vm-frontend-{lhr,nyc}{1-10}.somecloud.com"
      linux { 
         services.server {
            include "/stdlib/linux/service.conf"
         }
         debian.control.Depends = "postgres (>= 14)"
      }
    }

Then you run "conveyor push" from any OS and it downloads a Linux JVM, minimizes it for your app, bundles it with your JARs, produces a DEB from that, sftps it to the server, installs it using apt, that package integrates the server with systemd for startup/shutdown and dynamic users, it healthchecks the server to ensure it starts up and it can do rolling upgrades/downgrades. And of course the same for Go or NodeJS or whatever other server frameworks you like. So the idea is that if you use a portable runtime you can develop locally on macOS or Windows and not have to deal with Linux VMs, but deployment is transparent.

SystemD can also run containers and "portable services", as well as using cgroups for isolation and sandboxing, so there's no need to use debs specifically. It just means that you can depend on stuff that will get upgraded as part of whole OS system upgrades.

We use this to maintain our own servers and it's quite nice. One of the things that always bugged me about Linux is that deploying servers to it always looks like either "sftp a tarball and then wire things up yourself using vim", or Docker which involves registries and daemons and isolated OS images, but there's nothing in between.

Not sure whether it's worth releasing though. Docker seems so dominant.

[ Parent | on HN ]

Re:Rails on Docker · Fly by jrochkind1 on Thursday January 26, 2023 @01:52PM (#34536737)

> It's not handled.

So... what do actual real people do in practice? I am very confused what people are actually doing here.

LOTS of people seem to have moved to this kind of docker-based deploy for PaaS. They can't all just be... ignoring security patches?

I am very confused that nobody else seems to think the "handle patches story" is a big barrier to moving from heroku-style to docker-style... that everyone else just moves to docker-style? What are they actually doing to deal with patches?

I admit I don't really understand your solution -- I am not a sysadmin! This is why we deploy to heroku and pay them to take care of it! It is confusing to me that none of the other PaaS competitors to heroku -- including fly.io -- seem to think this is something their customers might want...

[ Parent | on HN ]

Re:Rails on Docker · Fly by brundolf on Thursday January 26, 2023 @02:09PM (#34537050)

It seems not that hard to just manually bump your version periodically, right? I get that it's not as nice as having it taken care of for you, but it's not like you're having to manage a whole system by hand

[ Parent | on HN ]

Re:Rails on Docker · Fly by jrochkind1 on Thursday January 26, 2023 @02:19PM (#34537220)

Assume I've never used Docker before, can you tell me what "manually bump your version periodically" means with regard to this question? Can anyone give me concrete instructions for what this looks like in a specific context? Like, say the first Dockerfile suggested in OP?

The Dockerfile has in it:

> ARG RUBY_VERSION=3.2.0

> FROM ruby:$RUBY_VERSION

Which it says gets us "gets us a Linux distribution running Ruby 3.2." (Ubuntu?). The version of that "linux distribution" isn't mentioned in the Dockerfile. How do I "bump" it if there is a security patch effecting it?

Then it has:

> RUN apt-get update -qq && apt-get install -y build-essential libvips && \ ...

Then I just run `fly launch` to launch that Dockerfile on fly.io. How do I "bump" the versions of those apt-get dependencies should they need them?

[ Parent | on HN ]

Re:Rails on Docker · Fly by hakunin on Thursday January 26, 2023 @02:51PM (#34537746)

If you're basing on a lang-specific container like ruby, then it's the version of that container in the FROM line. Notice how ruby images come in various versions of OS (https://hub.docker.com/_/ruby). You can specify that as part of the FROM string. However, they also let you drop the OS part, and only specify ruby version. This will usually default to an image with the latest OS provided by that docker repo. Nothing to bump in this case, just occasionally rebuild your own image from scratch, instead of from cache, to make sure it downloads the latest base image.

[ Parent | on HN ]

Re:Rails on Docker · Fly by michaelsalim on Thursday January 26, 2023 @09:05PM (#34541585)

This right here. Most of the time you just need to rebuild your image. If the project is being actively developed and built, nothing to worry about. (Unless you pin to a very specific OS version of course).

If it's not, you just need to trigger a build every so often. Maybe this could be a feature PaaS offers in the future.

[ Parent | on HN ]

Re:Rails on Docker · Fly by mike_hearn on Friday January 27, 2023 @05:39AM (#34544620)

Docker caches the results of each command, so to "bump" the versions you have to trigger a rebuild of the whole image and tell it to toss the cache. So it'll re-run apt-get update and use whatever the latest stuff is as of the moment of rebuild.

There are a few problems with this, as noted elsewhere in the thread:

1. You have to do an uncached rebuild and repush all of your images, using some ad-hoc company specific process. There's nothing that can do this for you at the end points or service levels, because Docker images are meant to be immutable after build and don't come with the scripts or inputs used to build them.

2. The default is to use caching, so devs may not notice that they didn't refresh their base OS for a while.

3. You don't get notified when updates are available or applied. There's nothing like the unattended-upgrades package that comes with Debian normally, which will apply upgrades and then tell you what happened.

4. Because of (3) the latency is very high. There is no story (other than third party scanners) for getting notified about urgent upgrades. If there's another zero day in OpenSSL then with a standard Linux install you'll get patched as soon as a new package is released and your machines update, so pretty quick (a day or so). With Docker images, it'll get patched on an app-by-app basis if and when people get around to doing an uncached rebuild and repush of the image.

5. Kernel upgrades are a whole can of eels in container-world. People like to think of the container as being a self-contained OS but it isn't. There is a largely unstated and untested assumption that any Linux distro user space can run on any kernel version or configuration, regardless of whether the OS originally shipped in that configuration, and everything will just automatically do something sensible. Mostly this assumption is OK because servers are very simple, but it's not actually guaranteed by anything. A lot of people misunderstand the "stable Linux syscall interface" guarantees and what that means.

It's for reasons like this that I prefer the slightly older way of running real binaries that are exposed to the OS and which use OS specific packages. I configured unattended-upgrades and use LTS versions of the OS, so that security patches just stream in without me doing anything. There are a few downsides to this too:

1. You have to remember to restart your servers from time to time, or schedule it to happen on a regular basis. The OS will patch stuff on disk, but not in memory. The Debian security patching story is thus not really complete. Of course this is still better than with Docker!

2. You do need to understand at least a bit of Linux sysadmin. Enough to know how to ssh in as root, use apt-get and so on.

3. The tooling story is poor, hence my musings above about demand for something better. Without Docker, today you're going to be manually copying files to the server, having to learn systemd and how to start/stop/enable services, how to restart them on upgrades etc. That's why I've written something that does it all for you.

[ Parent | on HN ]

Re:Rails on Docker · Fly by subarctic on Thursday January 26, 2023 @06:56PM (#34540595)

I mean, every Dockerfile has a FROM some_base_image line at the beginning, and it refers to an image and an optional tag (which defaults to the `latest` tag if omitted). As far as I know, whenever you build your docker image, it will check for a new version of the base image and download that if needed. So e.g. if you have FROM ubuntu:20.04 at the top of your Dockerfile, and a new version of ubuntu:20.04 is uploaded to dockerhub, the base image will be updated the next time you run docker build, which shouldn't be too long if you're making deployments regularly.

[ Parent | on HN ]

Re:Rails on Docker · Fly by mixmastamyk on Thursday January 26, 2023 @11:08PM (#34542352)

Most places are building software regularly. Once a week update the base layer of the container to the latest OS. Then update your libs and app. Run tests, deploy.

[ Parent | on HN ]

Re:Rails on Docker · Fly by lmm on Friday January 27, 2023 @01:37AM (#34543286)

> LOTS of people seem to have moved to this kind of docker-based deploy for PaaS. They can't all just be... ignoring security patches?

They can and they are, IME. I've seen images that literally run some version of Alpine Linux from 3 years ago.

[ Parent | on HN ]

Re:Rails on Docker · Fly by mike_hearn on Friday January 27, 2023 @04:46AM (#34544311)

"I admit I don't really understand your solution -- I am not a sysadmin! This is why we deploy to heroku and pay them to take care of it!"

Right. I didn't explain it very well then.

Basically it's a heroku-ish solution but as a tool rather than a service, and where you build locally instead of pushing source code to some remote cloud. You say "here's my build system, go push to these plain Linux VMs". Now, someone needs at least a bit of sysadmin knowledge - you need to know how to obtain Linux VMs, set up access to them, and make sure they're (self-)applying security updates. From time to time you'll need to roll to new OS releases and do restarts for kernel fixes. But that stuff isn't all that hard to learn.

Still - I'd be curious to know where your threshold is for touching Linux. With Heroku you never did, right? Dynos could have run Windows for all you knew? If you had a tool that e.g. you gave your cloud credentials to, and it then spun up N Linux VMs, logged in, configured automatic updates and some basic monitoring then let you push servers straight from your git repo, would you be interested in that? How much did you rely on Heroku support going in and helping you fix app-specific problems live in production?

[ Parent | on HN ]