How CircleCI Processes 4.5M Builds per Month

Re:How CircleCI Processes 4.5M Builds per Month by justinwp on Thursday July 13, 2017 @05:22PM (#14765362)

Some of our builds are running on CircleCI but have been trying out drone for a more container focused build. Been enjoying the flexibility and increased speed from custom build images which are trivial to make.

[ Parent | on HN ]

Re:How CircleCI Processes 4.5M Builds per Month by clarkdave on Thursday July 13, 2017 @10:33PM (#14766998)

If you don't mind tinkering, I highly recommend Buildkite. They provide the management & cloud UI and you run the actual build agents on your own infrastructure.

When you get it going it blows everything else away in value for money. We routinely have hundreds of containers on-the-go and as many concurrent builds as we need; this all runs on a large AWS spot instance at a cost of about $50 a month. Added bonus: our docker cache is always available.

Buildkite provide a CloudFormation stack, but we just opted to run the agents as containers via ECS to make the setup quicker and managing them easier.

[ Parent | on HN ]

Re:How CircleCI Processes 4.5M Builds per Month by Snappy on Monday July 17, 2017 @06:28PM (#14792766)

FWIW, GitLab CI also has an agent architecture so you can run the build agents (which we call runners) on your own infrastructure, on Linux, macOS, Windows, FreeBSD, Linux ARM, Docker, etc.

[ Parent | on HN ]

Re:How CircleCI Processes 4.5M Builds per Month by robertely on Friday July 14, 2017 @12:49AM (#14767542)

I'd love to hear more about using Buildkite. It appears to to offer greatly flexible agents with out a heavy cluster to manage or poorly written plugins to deal with...

[ Parent | on HN ]

Re:How CircleCI Processes 4.5M Builds per Month by clarkdave on Friday July 14, 2017 @06:22AM (#14768743)

Happy to answer any questions about it. We've been using Buildkite for a while - it has been a pleasant experience and has let us craft a CI setup whose cost would be prohibitive to us otherwise.

Another similar option we tried is AWS CodeBuild which has per-minute billing and provisions the build machines for you. However, it's very bare-bones and because you always get a from-scratch instance for each build you have to distribute your docker cache which is not ideal.

[ Parent | on HN ]

Re:How CircleCI Processes 4.5M Builds per Month by joevandyk on Friday July 14, 2017 @01:35AM (#14767748)

You are using a large spot instance on ecs?

[ Parent | on HN ]

Re:How CircleCI Processes 4.5M Builds per Month by clarkdave on Friday July 14, 2017 @06:03AM (#14768700)

Yes, it works very well. Earlier this year AWS automated the process of setting up a cluster backed by a spot fleet[0]. It's a very cost effective way to run lots of containers.

We actually use a combination of on-demand and spot container instances in production to keep costs down; we have some logic to provision more on-demand instances in case of multiple spot outages.

[0] https://aws.amazon.com/blogs/compute/powering-your-amazon-ec...

[ Parent | on HN ]

Re:How CircleCI Processes 4.5M Builds per Month by dtech on Thursday July 13, 2017 @05:52PM (#14765604)

CircleCi 2.0 left beta last week and is fully docker based

[ Parent | on HN ]

Re:How CircleCI Processes 4.5M Builds per Month by tedmiston on Thursday July 13, 2017 @08:55PM (#14766641)

They also provide a set of official pre-built Docker images for each language so 2.0 is easy to use even if your project doesn't use Docker.

https://circleci.com/docs/2.0/circleci-images/

[ Parent | on HN ]

Re:How CircleCI Processes 4.5M Builds per Month by timdorr on Thursday July 13, 2017 @07:55PM (#14766337)

Wercker is great, but the price jump above the free tier is a bit steep.

[ Parent | on HN ]

Re:How CircleCI Processes 4.5M Builds per Month by ergo14 on Thursday July 13, 2017 @05:32PM (#14765439)

I feel like http://concourse.ci/ might be the sweet spot here.

[ Parent | on HN ]

Re:How CircleCI Processes 4.5M Builds per Month by manishas on Friday July 14, 2017 @12:19AM (#14767432)

Have you tried Shippable? It's very focused on Docker based workflows for CI and also for building images, provisioning infrastructure, etc and then connecting all these jobs into a dependency matrix with configurable triggers.

https://www.shippable.com

[ Parent | on HN ]