This is a cool project and yay nix, but:

> Inside a container the file system is very slow

Because you are not using "a container".

You are using a container that happens to be running in a linux VM on your OS X laptop. It's not the containers fault, it's the entire virtual machine that you are running because you're trying to run technology built on top of one operating system while running a completely different one.

This. Put another way: it's slow only because you're on macOS and not on Linux.

That's why Linux needs more love.

I find it sad that many people use Docker, but the majority of them run Linux in a VM inside an expensive proprietary platform to do so, sometimes without even realizing it.

Meanwhile, the Linux desktop ecosystem is deteriorating and should be used more and receive more financial support.

Totally fair.

Overlay copy-on-write filesystems are slow. I think most container runtimes use them.

compared to full-VMs-pretending-to-be-containers everything is fast.

it seems like everybody here is using Macs for development to the point that if you don't say you aren't, you are assumed to be on a mac. Windows with WSL2 is actually pleasant to use, I can recommend trying it out. while WSL2 is a VM technically, the level of integration makes it basically native (if you use the VM disk for your workspace, which you should.)

> if you use the VM disk for your workspace, which you should.

Yep, if you try to build something that is stored on /mnt/c you'll have the same horrible performance.

It's not the COW filesystem that is slow, it's bind mounting the host filesystem into the VM. This ends up using something like 9p or sshfs.

What I really want is a tool like docker compose that builds on top of a tool like this (instead of docker)

Assuming you want the service management aspect of docker compose look at user-mode process supervisors like supervisord. You can install it in your nix environment and config all the dependent services it should manage. It's a little more crufty vs. the simplicity of docker compose but it also has a lot more power and flexibility. There are tons of other options to explore too like systemd user services or skarnet's s6 suite.

What I really want is to be able to do things like spin up an isolated postgres instance to run my unit tests against, with zero config on my part except specifying a postgres version in my "docker-compose.yml" equivalent.

+1

File systems inside the container aren't slow unless you're running on Windows or some other virtualization platform that makes it slow.

So, that’s what this is?

I haven't run devbox, but the README says that it requires docker to be installed. This makes me believe it isn't running the programs on my system (macos), but rather in the Linux VM.

It will use Nix to build a shell on your local machine directly, so you can run the program on your system directly (not in a VM).

It also has some early functionality where you can turn that shell into a Docker container, so you can run the same shell + program in other environments or in the cloud

so an abstration on top of docker?

It uses the Docker CLI to build container images out of the Nix shells it creates, if you ask it to export one to a Docker image for you. Otherwise it just uses Nix locally, no virtualization.

I wonder if it takes this approach because there's some issue with using Nixpkgs' dockerTools on macOS— those tools let you create Docker/OCI images without even having Docker installed.

https://nix.dev/tutorials/building-and-running-docker-images... asserts that you'd need a remote builder or cross-compiling. So it seems possible (unless maybe there are still enough cross-compile issues?)

Can cross builds hit the main binary caches or is that awaiting Hydra support for content-addressed derivations? Are there big caches for cross-compiled packages?

I'm guessing awaiting based on how much rebuilding I had to do yesterday.

docker is not virtualization either

1. Yes it is— there are many kinds of virtualization other than machine virtualization. Containers are a form of operating system virtualization.

2. Docker on non-Linux also requires machine virtualization; the `docker` CLI connects to a `dockerd` daemon running on a virtual machine.

The question I posed earlier is motivated by understanding this and knowing that on macOS, the binaries that devbox runs on your local machine are completely different (of a different architecture, even) than the binaries that devbox will pack into a Docker image. Since it uses Nix to build them, it either needs to run Nix in a virtual Linux guest (which it does using the guest that Docker for Mac sets up) or have its Mac-bound Nix build Linux binaries.

Using `docker buildx` obviates the need to set up devbox's own Linux VM to target as a remote builder, or, alternatively, cross compile the required Nix packages. But it comes at the cost of adding a dependency on Docker even though devbox doesn't use the Docker runtime for its shells

No, it just creates images compatible with docker.