Initially we did a lot of cloning from source and compiling/installing dependencies, but it's very slow, there's a lot of wasted time in rebuilding identical code, and it's hard to provide patches and upgrades to customers.
I guess I just don't see the time vs. effort value in optimizing most smaller projects.
For example, that 75 gem project may take 5 minutes to build once but after that it takes 10 seconds to build and push a version that updates the app code.
I'm ok with this pattern for most of my projects because you can easily get by with 1 host to serve tens of thousands of requests a month on most typical projects. It's not like I'm spinning up and destroying dozens of instances a day where the network overhead is a legit concern (if I were, then I would optimize).
Is the common procedure in the Docker world to build an application image that includes all the build tools that were used to build native dependencies? That seems like it does generate a pretty large image.
I figured I'd take a three-step approach to my first node.js app in Docker:
1. Build an image to build my dependencies. This uses the same base image as step #2 will, but installs all the development tools and libraries (eg. build-essentials, libpq5-dev), and then outputs a .tar.gz to a shared volume containing my node_modules folder.
2. Build an image with my dependencies; imports the runtime versions of any libraries (eg. libpq5), imports & expands the .tar.gz generated by #1.
3. Build an image with my application, FROM the image in #2.
The process is optimized by having the automation check for the existence of #2 by hashing the contents of the relevant Dockerfiles, and the package.json list of dependencies, and doing a `docker pull` with that hash to see if I've already built #2. If so, my build just needs to build #3.
It's a bit more complex (Hello, everything in Docker-land), but ends up being pretty powerful. But your post makes me think I've over-complexified it a bit.