adityamaru

Have you checked out https://docs.blacksmith.sh/docker-builds/incremental-docker-...? This should help setup a shared, persistent docker layer cache for your runners

thanks for clarifying!

> Nix jobs run inside rootless Podman containers but the Nix store and Nix daemon socket are passed through from the host

That's a neat idea, was the primary motivation for building this out the perf gains on the table?

This is a common cause yeah but becoming less of an issue with increasing support for ARM runners.

indeed, thats a good callout. We'll add this to our README over at https://github.com/useblacksmith/remote-buildkit-terraform

`hope` is a good name for a service trying to solve this problem :D

> 30 minute docker builds?

At Blacksmith we do see this pretty often! Rust services in particular are the most common offender.

What provider do you prefer? We're big fans of Hetzner

do you self host your jenkins deployment in your AWS account?

totally agree, github actions has done an excellent job at this lowest layer of the build pipeline today but is woefully inadequate the minute your org hits north of 50 engineers

oh TIL, that is interesting

Yeah that still holds true to some extent today with the GHA cache. Blacksmith colocates its cache with our CI runners, and ensures that they're in the same local network allowing us to saturate the NIC and provide much faster cache reads/writes. We're also thinking of clever ways to avoid downloading from a cache entirely and instead bind mount cache volumes over the network into the CI runner. Still early days, but stay tuned!

This is exactly the sort of insight that led us to work on Blacksmith. Since we own the hardware we run CI jobs on there are some exciting things we can do to make these "sticky disks" work the way you describe it. Stay tuned!