News for hackers. Stuff that matters.
Re:GitHub now provides Arm-based runners by joshstrange on Wednesday June 05, 2024 @08:30AM (#40584057)
GitHub’s runners are a joke. Switching to WarpBuild was the best decision I made for my CI/CD. I halved my build time and got cheaper minutes.

GH’s macOS runners particularly were complete trash.

Re:GitHub now provides Arm-based runners by jjice on Wednesday June 05, 2024 @11:28AM (#40586095)
We're a small team (four engineers) and we purchased four Dell mini PCs to toss into our small office space. The break even compared to our AWS hosted runners (which were much faster and cheaper than the GH ones) was like four or five months, plus they cut the time again by over half. We have to deal with them if something weird happens, but it's been such a low time investment that it has been such a great choice.

Plus, who doesn't love having some new hardware to play with?

Re:GitHub now provides Arm-based runners by edweis on Wednesday June 05, 2024 @11:59AM (#40586562)
Why 4 Dell Minis? You don’t need a monitor for CI/CD, and why 4? Is it for redundancy? Performance?

We are interested in doing the same setup.

Re:GitHub now provides Arm-based runners by candiddevmike on Wednesday June 05, 2024 @01:00PM (#40587424)
Not OP, but I went with MOREFINE AMD Ryzen mini PCs. They can be configured with a ton of CPU/RAM (including ECC). They're really compact, support 2.5 Gb ethernet, and are great to work with. A comparable Dell workstation with ECC RAM would've been significantly more expensive, and you have to deal with their business folks to purchase them (or at least I couldn't figure out how to buy them direct).
Re:GitHub now provides Arm-based runners by jjice on Wednesday June 05, 2024 @02:00PM (#40588193)
Wasn't my choice, but one wasn't enough so we threw a few more at it. So mostly for performance, but the redundancy is an added benefit. They were a pretty decent space, power/heat, performance, and noise ratio for us.

Our office is a single room with six desks, so space, noise, and heat are a larger consideration for us. I use a mini PC at home for a personal server as well, but that's just because used enterprise equipment is often a great deal.

Re:GitHub now provides Arm-based runners by dehrmann on Wednesday June 05, 2024 @12:01PM (#40586593)
> We have to deal with them if something weird happens

Check back in three years once you've had to deal with OS upgrades, power outages, running out of disk space, and one of them failing.

Re:GitHub now provides Arm-based runners by abuani on Wednesday June 05, 2024 @12:04PM (#40586627)
Three years sounds like a reasonably good return on investment. At that point the teams either big enough to justify paying $$ for a hosted solution, or still cheaper to buy dedicated hardware.
Re:GitHub now provides Arm-based runners by delfinom on Wednesday June 05, 2024 @12:17PM (#40586805)
You can also just rent dedicated servers from hetzner or ovh for far cheaper than AWS instances with more horsepower. More expensive than a dell mini in a closet. But you get datacenter Internet and power backup.
Re:GitHub now provides Arm-based runners by re-thc on Wednesday June 05, 2024 @12:51PM (#40587309)
> justify paying $$ for a hosted solution, or still cheaper to buy dedicated hardware.

Always cheaper if you have at least some demand. For the lazy fix even just get a few mac mini and it'd work fine.

Re:GitHub now provides Arm-based runners by delfinom on Wednesday June 05, 2024 @12:12PM (#40586736)
It's a CI runner my dude.

You just obliterate and reinstall the base system in what 10 minutes?

Re:GitHub now provides Arm-based runners by cr125rider on Wednesday June 05, 2024 @12:27PM (#40586928)
Are any of those problems challenging to deal with? Even if it’s a few expensive man hours the ROI is still quite short.
Re:GitHub now provides Arm-based runners by barkingcat on Wednesday June 05, 2024 @01:24PM (#40587755)
if the break even was 4-5 months, it means compared to going to cloud vendors, in 36 months, the self hosted solution would have saved 7.2 times the cost!!

Even if within 3 years there needs to be OS upgrades, you can just e-cycle the old machines and buy brand new minipc's (with added benefit of increased performance) every time you need to upgrade, and you'd still be ahead!!

Re:GitHub now provides Arm-based runners by jjice on Wednesday June 05, 2024 @01:57PM (#40588155)
We're about eight months in at the moment, so here are some current stats:

- No OS upgrades (running Ubuntu 22 LTS), but regular update

- We've had a power outage, but not during the day, so when they rebooted, they went right back to action. Same issue with a network outage in the office.

- We have hit disk space issues due to not clearing all the data we needed to. A quick SSH and an rm or two did the job, plus another 30 minutes to an hour to update a cron to remove that portion of the cache

I see what you mean for sure, and there will undoubtedly be more things that come up at some point, but with the savings, it would take quite a few engineering hours per month to lose the savings at this point.

And I won't lie, when something goes wrong, it's a nice change of pace to deal with a server instead of code once and a while. Can't actually use that for calculations, but I don't hate when it happens so far.

Re:GitHub now provides Arm-based runners by rcleveng on Wednesday June 05, 2024 @07:28PM (#40591812)
When a better ROI of 4-5 months, you could just buy a new one when any of those things happen, and still be net positive.
Re:GitHub now provides Arm-based runners by bravetraveler on Thursday June 06, 2024 @07:55AM (#40596334)
> Check back in three years once you've had to deal with...

This being routine, I can tell you right now how it will go. Let's start!

> OS upgrades

Distinction needed: update/upgrade.

The first is entirely automatable, some may say table stakes. This is CI - take your pick, pull vs push deployment model. Decide to update in this job or another.

Upgrades are a common problem for either hosting world. Can my software work in this new environment? Hence, CI.

> power outages

Set the firmware to return to last power state. Also buy a couple UPS' and a PDU. Fixed. The provider and networks between going away isn't so easy.

> Running out of disk space

How is this unique for them to solve? Hosted workers/cloud servers have finite disks too.

Clean your workspaces in your pipelines.

> and one of them failing.

See deployment model again. Someone else in the thread said it best, these are CI runners man.

Re:GitHub now provides Arm-based runners by candiddevmike on Wednesday June 05, 2024 @10:42AM (#40585398)
Or just run your own runners on-premise. You'd be amazed at how fast builds can be with OTS hardware, or even something like a laptop or Mac mini.
Re:GitHub now provides Arm-based runners by joshstrange on Wednesday June 05, 2024 @12:00PM (#40586580)
I considered this for my personal business but the minutes are stupid cheap and to pay off a Mac mini it would take me many, many years to break even. For a company with 10's-100's-1000's+ employees writing code and needing CI/CD the math will be different. I came close to buying a Mac Mini anyways, costs be damned, to get away from GH's slow runners.
Re:GitHub now provides Arm-based runners by dehrmann on Wednesday June 05, 2024 @12:04PM (#40586630)
> minutes are stupid cheap

At smaller scales, as long as your builds aren't stupud-slow, cloud providers should be cheaper just because they can utilize their hardware more (10x?) efficiently.

Re:GitHub now provides Arm-based runners by joshstrange on Wednesday June 05, 2024 @12:44PM (#40587191)
Totally agree, and I called out that my scale is small so cloud makes a lot more financial sense for me (especially as a solo founder, I don't want to manage more hardware than I have to already).
Re:GitHub now provides Arm-based runners by duped on Wednesday June 05, 2024 @12:02PM (#40586601)
GitHub Actions is a joke. It baffles me that they've been breaking file permissions on upload for years and they haven't fixed it yet. God forbid you want to ship an executable in a tarball or save it for a later stage of your build.
Re:GitHub now provides Arm-based runners by aromeronavia on Wednesday June 05, 2024 @10:46AM (#40585461)
Can't agree more with this, I recently migrated my company to use WarpBuild and our CI got 50% faster, not even kidding. We were using self-hosted runners which were not as efficient as we thought so we spawned EC2 instances and connected them to the github network to be runners, but we had such an amount of issues that we wanted to go back to non-self hosted. At the end a tech lead of the company suggested using Warp and we could not be happier with the results.

Also the CEO Surya is such a great guy, we had a small networking issue that lasted for about two hours and they gave us credits for all the runners affected with that, and their customer support is blazingly fast.

Re:GitHub now provides Arm-based runners by suryao on Wednesday June 05, 2024 @10:51AM (#40585531)
Thank you so much for the kind words!
Re:GitHub now provides Arm-based runners by crohr on Wednesday June 05, 2024 @12:29PM (#40586957)
> We were using self-hosted runners which were not as efficient as we thought so we spawned EC2 instances and connected them to the github network to be runners, but we had such an amount of issues that we wanted to go back to non-self hosted

For those hitting that same kind of issue (and you are many), you should check out my product https://runs-on.com. 1-click install, 1-click upgrades, and from 7x to 16x cheaper runners on AWS.

Re:GitHub now provides Arm-based runners by threecheese on Wednesday June 05, 2024 @07:07PM (#40591690)
Brilliant idea!! I just dug through your docs. You significantly undercut GitHub on runner pricing, using your customers’ infrastructure for both runners and their orchestration, which is managed by your software, some cloudformation, and amis that you maintain and license in return for a small slice of that savings. I love it, and if our cloudops org ever threatens to charge back CI to our budget I’ll be a customer.
Re:GitHub now provides Arm-based runners by joshstrange on Wednesday June 05, 2024 @01:38PM (#40587917)
I was deploying to CloudFormation and I thought that was the slow part but my "build" times went from 20min to 10min. My iOS builds also were halved.
Re:GitHub now provides Arm-based runners by 999900000999 on Wednesday June 05, 2024 @09:36AM (#40584587)
I understand the Mac builds, but the entire appeal of GitHub runners is it's literally built in. I can just add a YAML file, and we have CICD!
Re:GitHub now provides Arm-based runners by joshstrange on Wednesday June 05, 2024 @09:39AM (#40584619)
I get that, but you can run GH runners on your own hardware or someone else's. Switching to WarpBuild was literally as easy as linking my GH account and changing the running image from `macOS-latest` to `warp-macos-14-arm64-6x` and everything worked. Secrets and everything.
Re:GitHub now provides Arm-based runners by palata on Wednesday June 05, 2024 @03:31PM (#40589410)
> Secrets and everything.

I guess you had to import the secrets, too...

Re:GitHub now provides Arm-based runners by yohannparis on Wednesday June 05, 2024 @09:40AM (#40584633)
Different needs, different tools! I work on a open-source project, and not having to pay for CI/CD with a "simple" yml file configuration is simpler and easy to manage.
Re:GitHub now provides Arm-based runners by 01HNNWZ0MV43FF on Wednesday June 05, 2024 @09:42AM (#40584647)
It's only free if your time is worthless :P every couple weeks I have to debug something that I can't replicate locally because I don't have local CI
Re:GitHub now provides Arm-based runners by joshstrange on Wednesday June 05, 2024 @09:44AM (#40584672)
That's the other plus with something like WarpBuild, I haven't used it yet but you can pause the build and SSH in:

https://github.com/WarpBuilds/action-debugger

Re:GitHub now provides Arm-based runners by EionRobb on Wednesday June 05, 2024 @03:16PM (#40589140)
Have you tried using act for that? https://github.com/nektos/act Has saved me a bunch of CI debug pain.
Re:GitHub now provides Arm-based runners by vessy-st6io on Wednesday June 19, 2024 @03:48AM (#40725894)
You can try FlyCI Wingman - an AI agent that analyzes and fixes your failing builds in GitHub actions. We're about to release it tomorrow, so feel free to subscribe and be among the first to try: https://www.flyci.net/ .
Re:GitHub now provides Arm-based runners by joshstrange on Wednesday June 05, 2024 @09:44AM (#40584666)
If you are open source and getting free minutes then it really doesn't matter how long builds take.

I'm still using GH Actions, just not their minutes.

Re:GitHub now provides Arm-based runners by suryao on Wednesday June 05, 2024 @10:31AM (#40585250)
There is a lot of inefficiency in CI today and we're out to solve it.

Thanks for the WarpBuild love!

Re:GitHub now provides Arm-based runners by KRAKRISMOTT on Wednesday June 05, 2024 @11:56AM (#40586516)
How do they compare to Ubicloud?

https://www.ubicloud.com/

Re:GitHub now provides Arm-based runners by suryao on Wednesday June 05, 2024 @12:06PM (#40586656)
(WarpBuild founder here) We have had a lot of users evaluate multiple tools including ubicloud but chose us. Typically, they've mentioned our runners being faster, our caching is very useful, and queueing delays being lower.

However, my last anecdata was from a few weeks ago. You can try us out and take a call for yourself.

Our focus is on CI runners and efficiency and I think that gives us an edge on the devex. Ubicloud is definitely cheaper though.

Re:GitHub now provides Arm-based runners by joshstrange on Wednesday June 05, 2024 @11:59AM (#40586557)
I think I skipped over evaluating ubicloud because they don't have macOS runners. There was another place I liked but I needed macOS runners they didn't have them either.
Re:GitHub now provides Arm-based runners by ugh123 on Wednesday June 05, 2024 @01:08PM (#40587549)
How do you handle extremely large repos? Can these be cached on machines, per-customer?
Re:GitHub now provides Arm-based runners by suryao on Wednesday June 05, 2024 @01:29PM (#40587815)
Assuming the question was reg. WarpBuild - we provide blazing fast caches with unlimited sizes. These still need to be downloaded to machines and setup on each workflow job.

We don't (yet) have support for pre-pulled repos on runners per customer. Are the large repo sizes despite fetching only the refs required for the specific context [1][2] which can help speed things up significantly.

[1] https://www.atlassian.com/git/tutorials/big-repositories [2] https://stackoverflow.com/questions/48072080/how-to-manage-l...

Re:GitHub now provides Arm-based runners by electroly on Wednesday June 05, 2024 @01:39PM (#40587933)
Ditto. I came for the cheaper Apple Silicon runners (before GitHub Actions had them available for free) and stayed for the faster runs across the board. I cut a 24 minute build down to 10 minutes just by switching over to WarpBuild runners. Not a shill, just a happy customer.
Re:GitHub now provides Arm-based runners by suryao on Wednesday June 05, 2024 @01:44PM (#40587995)
And I a happy provider - thank you!