dan_manges

(Disclosure: I'm a cofounder of RWX) For anybody wanting to solve similar problems and considering Bazel, take a look RWX. It's built around the same concepts of content-based caching and graph-based task execution, but it's far more runtime agnostic and easier to adopt. https://rwx.com

I've been working on this problem for the past couple of years. State of the art:

- local CLI instead of git push to run

- graph-based task definitions with automatic distributed execution, instead of the job/step abstraction

- automatic content-based caching to skip unnecessary executions (happens a lot in CI pipelines)

- container-based runtime (instead of proprietary base images) without using docker directly (too slow)

There are a lot of other ways to improve the developer experience. Happy to chat with anybody interested, I'm [email protected]

Hey! I'm one of the RWX cofounders. We don't have a self-hosted option yet, but we're interested in opening it up in the future. Our infrastructure is more complex to run than normal self-hosted runners infrastructure, due to extensive caching and provisioning optimizations.

We support anchors in our CI/CD syntax at RWX, but only in a specific `aliases` section. I think this is a nice compromise.

  aliases:
    common-env: &common-env
      key1: value1
      key2: value2

  tasks:
    - key: some-task
      run: ...
      env:
        <<: *common-env

https://www.rwx.com/docs/mint/aliases

There's a difference between small scale CI and large scale CI.

Small scale: a project is almost small enough to run the build and tests locally, but you still want to have a consistent environment and avoid "works on my machine" problems.

Large scale: a project is so large that you need to leverage remote, distributed computing to run everything with a reasonable feedback loop, ideally under 10 minutes.

The opposite ends of the spectrum warrant different solutions. For small scale, actually being able to run the whole CI stack locally is ideal. For large scale, it's not feasible.

> A CI system that’s a joy to use, that sounds like a fantasy. What would it even be like? What would make using a CI system joyful to you?

I spent the past few years building RWX[1] to make a CI system joyful to use for large scale projects.

- Local CLI to read the workflow definitions locally and then run remotely. That way can you test changes to workflow definitions without having to commit and push.

- Remote breakpoints to pause execution at any point and connect via ssh, which is necessary when running on remote infrastructure.

- Automatic content-based caching with sandboxed executions, so that you can skip all of the duplicative steps that large scale CI otherwise would. Sandboxing ensures that the cache never produces false positives.

- Graph-based task definitions, rather than the 1 job : 1 VM model. This results in automatic and maximum parallelization, with no redundancy in setup for each job.

- The graph based model also provides an improved retry experience, and more flexibility in resource allocation. For example, one task in the DAG can crank up the CPU and memory without having to run more resources for downstream tasks (steps in other platforms).

We've made dozens of other improvements to the UX for projects with large build and test workflows. Big engineering teams love the experience.

[1] https://rwx.com

RWX | Software Engineer | REMOTE (US) | Full-time | https://rwx.com

$175K – $250K + equity

We're a CI/CD platform with an entirely differentiated execution model. Rather than parallelizing jobs on VMs, we provide a serverless interface using a graph of tasks. We then have automatic, content-based caching, which eliminates the large amount of redundant executions that CI platforms have today. You can test changes from a local CLI so that you don't have to commit and push to run. We've made substantial improvements to every aspect of the developer experience.

We're looking for backend-focused engineers who excel at building developer tools. Our engineers are effectively product managers as well and work highly autonomously.

How to apply: email me at [email protected]

GitHub Actions should use a lockfile for dependencies. Without it, compromised Actions propagate instantly. While it'd still be an issue even with locking, it would slow down the rollout and reduce the impact.

Semver notation rather than branches or tags is a great solution to this problem. Specify the version that want, let the package manager resolve it, and then periodically update all of your packages. It would also improve build stability.

Checksums are a good use case, where you’re not worried about any sort of forged collisions and you want to calculate the checksum as fast as possible. Although cryptographic hashes are fast enough that there likely aren’t too many use cases where the difference in speed is meaningful.

We're solving a lot of these problems with Mint: https://rwx.com/mint

Key differentiators:

* Content-based caching eliminates duplicate execution – only run what's relevant based on the changes being made

* Filters are applied before execution, ensuring that cache keys are reliable

* Steps are defined as a DAG, with machines abstracted away, for better performance, efficiency, and composition/reuse

A new CI/CD platform. Local CLI (run without git push), remote debugger, automatic content-based caching, DAG-based definition, and dynamic tasks. It’s taken a lot of work to build it, but I’m really excited about how well it’s working. https://rwx.com/mint

I don’t understand why any of that indicates that a 70 year old couldn’t have created this, unless you think 70 year old people aren’t capable of doing that.

Solving those problems would be a dream. Although the interface to initiating a CI/CD run can be solved without solving those other parts. Allowing the run definitions to be passed in, and ensuring any data based on the version control event is parameterized, is an effective solution to the push-to-run problem.

Root | Columbus, OH | Full-time | Onsite | $80k-$150k | https://joinroot.com

Root is an auto insurance carrier, like GEICO and Progressive. We use data science to identify and insure good drivers, reducing insurance premiums for good drivers significantly as a result.

We're a startup — we're 60 people who have been working on this since March 2015. We've built an iOS and Android app that gathers data on how well people drive. We use that to set insurance prices. To build the best possible product and user experience, we went through the arduous process of starting an insurance carrier from scratch.

We're focused on becoming a national insurance carrier and are now live in eight states.

We raised a $7M Series A from Drive Capital in 2015 and we're looking to bring on a couple more talented engineers.

Tech stack involves Ruby / Rails and Javascript / React Native.

Email us at [email protected] to apply and we'll respond to you promptly.

We don't right now. We're a car insurance carrier that uses our mobile app to gather data as people drive. Since all of our customers have our app installed, we haven't looked to building a web app version yet. Although we could reuse some logic on the web, the overlap would be minimal. Most of the code is centered around the UI, and a web app would likely have a very different interface. It would be nice to share some business logic functions, although we tend to not put much of that in the client anyway. Most of it happens server-side, with client hitting APIs to check business rules.

We have a 15-person engineering team and converted our mobile app to React Native at the beginning of this year. We do not regret it. Our pace of delivery is much faster with React Native than what we were able to achieve building natively. Our app has over 100 screens, and only needing to implement features once to have them on both platforms is nice. The feedback loop when making code changes is much faster, especially when writing unit tests. There are some rough edges, but the benefits far outweigh the costs for us.

Our iOS app: https://itunes.apple.com/us/app/root-car-insurance/id1021256...

Our Android app: https://play.google.com/store/apps/details?id=com.joinroot.r...

Root | Columbus, OH | Full-time | Onsite | https://joinroot.com

Root is an auto insurance company, like GEICO and Progressive.

We use data science to identify and insure good drivers, reducing insurance premiums for good drivers significantly as a result.

We're a startup — we're 14 people who have been working on this for a little over a year. We've built an iOS app that gathers data on how well people drive. We use that to set insurance prices. To build the best possible product and user experience, we went through the arduous process of starting an insurance carrier from scratch.

We've raised $7M, and we're looking to bring on a couple more talented engineers. Tech stack involves Ruby / Rails, iOS / Swift, and Javascript / React Native. Email me at [email protected]