Re:Faster Docker builds using a remote BuildKit instance
by throw1948012309 on Monday August 12, 2024 @09:27PM
(#41231275)
> My microservices all build and deploy in parallel so this system doesn't get slower as you expand to more services.
If you're deploying all your "microservices" in parallel, then what you might have built is a distributed monolith.
A microservice can be tested and deployed independently.
Re:Faster Docker builds using a remote BuildKit instance
by nine_k on Monday August 12, 2024 @09:33PM
(#41231307)
I don't see a contradiction. I read it that the microservices are independent and thus can build in parallel, if several teams work on changes to several microservices.
Spinning a build worker outright when a change us pushed is the fastest way, and may be expensive if the build process is prolonged.
OTOH I've seen much faster image build times, with smart reuse of layers, so that you don't have to re-run that huge npm install if your packages.lock did not change.
Re:Faster Docker builds using a remote BuildKit instance
by bhouston on Monday August 12, 2024 @09:32PM
(#41231299)
Whether it is a distributed monolith or a set of microservices is independent of the speed of build.
Re:Faster Docker builds using a remote BuildKit instance
by adityamaru on Monday August 12, 2024 @09:13PM
(#41231204)
> 30 minute docker builds?
At Blacksmith we do see this pretty often! Rust services in particular are the most common offender.
Re:Faster Docker builds using a remote BuildKit instance
by jeffparsons on Monday August 12, 2024 @09:34PM
(#41231311)
I'm working on an ungodly pile of hacks (https://github.com/jeffparsons/hope) to help with this. Coming Soon™: S3 backend and better tests.
Re:Faster Docker builds using a remote BuildKit instance
by adityamaru on Monday August 12, 2024 @09:38PM
(#41231335)
`hope` is a good name for a service trying to solve this problem :D
Re:Faster Docker builds using a remote BuildKit instance
by jeffparsons on Monday August 12, 2024 @09:54PM
(#41231408)
Haha, thanks. I chose it because:
- Here's One (I? You?) Prepared Earlier - Sometimes Hope _is_ a (caching) strategy - And yeah, I really hope I can make this thing work well.
I like silly puns. They bring me joy.
Re:Faster Docker builds using a remote BuildKit instance
by bhouston on Monday August 12, 2024 @09:25PM
(#41231263)
Yikes. I would be so much less productive with a 30 minute build time.
Re:Faster Docker builds using a remote BuildKit instance
by jjayj on Monday August 12, 2024 @09:48PM
(#41231374)
I regularly build images where we install Python from source, which makes 30m seem quite normal...
Re:Faster Docker builds using a remote BuildKit instance
by bhouston on Monday August 12, 2024 @09:54PM
(#41231405)
Why?
Re:Faster Docker builds using a remote BuildKit instance
by xyst on Monday August 12, 2024 @10:30PM
(#41231608)
Maybe to secure the supply chain
Re:Faster Docker builds using a remote BuildKit instance
by smallerize on Monday August 12, 2024 @11:51PM
(#41232040)
If they can't trust their own build cache, they really have problems.
Re:Faster Docker builds using a remote BuildKit instance
by jjayj on Tuesday August 13, 2024 @07:33AM
(#41234399)
We do development/builds in containers to make things easier on the devs. These are the Python build containers, so we have to rebuild every time there's a new version of Python or one of its dependencies.
I know it is out of style for some, but my microservice architecture, which has a dozen services, each takes about 1:30m to build, maybe 2m at most (if there is a slow Next.js build in there and a few thousand npm packages), and that is just on a 4 core GitHub Actions worker.
My microservices all build and deploy in parallel so this system doesn't get slower as you expand to more services.
(Open source template which shows how it works: https://github.com/bhouston/template-typescript-monorepo/act... )