crohr

Skip the middleman, just use AWS with runs-on.com (I'm the developer). Better instance choice, much cheaper, and you can setup an AWS budget alert.

Really nice, thanks!

I’m the founder of runs-on.com, we should talk!

So cool, thanks for sharing. Do you think it might be easier to maintain than something like `act` over the long run?

Love the observability you’ve built into it. But push a little bit further the scale and you’ll probably hit limits with the persistent self hosted runner approach, and end up with something like runs-on (runs-on.com)

> You know how I know GitHub’s runners are bad? Because there’s an entire cottage industry of companies whose sole product is “GitHub Actions, but the runners don’t suck.” Namespace, Blacksmith, Actuated, Runs-on, BuildJet

He's not wrong. Buildjet just announced they were shutting down though, citing recent improvements to the GitHub Actions platform.

For the record I maintain the Runs-on [1] he's talking about, as a solo developer.

[1] https://runs-on.com

For alternatives, I maintain a benchmark at https://runs-on.com/benchmarks/github-actions-cpu-performanc...

Not fully OSS, but you can have a look at https://github.com/runs-on/runs-on

Thanks for posting this :) runs-on.com domain posts never get traction, wonder if this could be due to YCombinator investment in 5+ startups in that space... Oh well.

I am developing a self-hosted solution for this [1]. It’s true that it’s somewhat of a pain but JIT runners allow a lot of flexibility that we don’t find elsewhere.

[1] https://runs-on.com

Probably long overdue, but per-minute price vs per-job is quite expensive. Wouldn’t like to be in the shoes of “only” 2x cheaper third parties. If they follow up with faster runners… interested to see if they ever come up with a good SDK for their scale set API, will integrate it in RunsOn!

But you can self-host github actions runners as well.

It is not transparent though, so it doesn't work with all the other actions that use the cache toolkit, and you have to reference a specific action.

Also note this open-source project that shows a way to implement this: https://github.com/falcondev-oss/github-actions-cache-server

At this point this is considered a baseline feature of every good GitHub Actions third-party provider, but nice to see the write-up and solution they came up with!

Note that GitHub Actions Cache v2 is actually very good in terms of download/upload speed right now, when running from GitHub managed runners. The low speed Blacksmith was seeing before is just due to their slow (Hetzner?) network.

I benchmarked most providers (I maintain RunsOn) with regards to their cache performance here: https://runs-on.com/benchmarks/github-actions-cache-performa...

FYI you can also use a local S3 bucket as cache backend, and get 5x faster bandwidth and unlimited storage compared to a runner managed by GitHub: https://github.com/runs-on/cache

What kind of CI runners do you use then? Do you self-host?

I think if you want this submission to work, you really need to work on your getting started guide. Currently it just says: "Clone the repo".

With Forge, how long does it take to start a runner once everything is setup?

I guess TL;DR just use ephemeral runners when self hosting? There are lots of solutions for that. Also would be nice for GitHub to do something on the security front (allowlist / blocklists if IPs, hosted, etc or at least just reporting on traffic)

It doesn't provide the on-demand price, nor the instance specs. Also requires authentication. With this API I can also ask for pricing across a whole family type (e.g. `c7a`).

I'm maintaining a benchmark of various GitHub Actions providers regarding I/O speed [1]. Depot is not present because my account was blocked but would love to compare! The disk accelerator looks like a nice feature.

[1]: https://runs-on.com/benchmarks/github-actions-disk-performan...

For me the main issue with managed GitHub runners is their low performance compared to the cost. In the context of developing my own tool for managing self-hosted runners, I maintain a benchmark here [1] that compares GitHub vs third party providers.

https://runs-on.com/benchmarks/github-actions-cpu-performanc...

"The world runs on Excel" - I think it's true and we should bring AI tooling inside the spreadsheet, not build yet another SaaS silo from where your data will never escape.

Also opens a lot of doors for integrating with private tools or data sources, especially with MCP tooling or such. I published a POC of what I mean at https://supersheets.ai/.

https://runs-on.com - Faster, 90% cheaper GitHub Actions runners!

You might be better served with a provider that uses AWS graviton processors though. I maintain a benchmark at https://runs-on.com/benchmarks/github-actions-runners/. I’ll add the new GitHub processor soon.