Re:How CircleCI Processes 4.5M Builds per Month
by Snappy on Monday July 17, 2017 @06:28PM
(#14792766)
FWIW, GitLab CI also has an agent architecture so you can run the build agents (which we call runners) on your own infrastructure, on Linux, macOS, Windows, FreeBSD, Linux ARM, Docker, etc.
Re:How CircleCI Processes 4.5M Builds per Month
by robertely on Friday July 14, 2017 @12:49AM
(#14767542)
I'd love to hear more about using Buildkite. It appears to to offer greatly flexible agents with out a heavy cluster to manage or poorly written plugins to deal with...
Re:How CircleCI Processes 4.5M Builds per Month
by clarkdave on Friday July 14, 2017 @06:22AM
(#14768743)
Happy to answer any questions about it. We've been using Buildkite for a while - it has been a pleasant experience and has let us craft a CI setup whose cost would be prohibitive to us otherwise.
Another similar option we tried is AWS CodeBuild which has per-minute billing and provisions the build machines for you. However, it's very bare-bones and because you always get a from-scratch instance for each build you have to distribute your docker cache which is not ideal.
Re:How CircleCI Processes 4.5M Builds per Month
by joevandyk on Friday July 14, 2017 @01:35AM
(#14767748)
You are using a large spot instance on ecs?
Re:How CircleCI Processes 4.5M Builds per Month
by clarkdave on Friday July 14, 2017 @06:03AM
(#14768700)
Yes, it works very well. Earlier this year AWS automated the process of setting up a cluster backed by a spot fleet[0]. It's a very cost effective way to run lots of containers.
We actually use a combination of on-demand and spot container instances in production to keep costs down; we have some logic to provision more on-demand instances in case of multiple spot outages.
[0] https://aws.amazon.com/blogs/compute/powering-your-amazon-ec...
When you get it going it blows everything else away in value for money. We routinely have hundreds of containers on-the-go and as many concurrent builds as we need; this all runs on a large AWS spot instance at a cost of about $50 a month. Added bonus: our docker cache is always available.
Buildkite provide a CloudFormation stack, but we just opted to run the agents as containers via ECS to make the setup quicker and managing them easier.