News for hackers. Stuff that matters.
Re:A Better Model for Stacked Pull Requests by bob1029 on Friday April 23, 2021 @07:30PM (#26920494)
I've considered building systems that would speculatively run builds using forward projections of all pending pull requests, as well as run multiple parallel scenarios on merge order + rebase such that merge conflicts are minimized. Maybe sprinkle in some monte carlo if the numbers get a little intense.

In practice, I feel it is much easier to assign work and organize your codebase so that logically-different business functions can be managed in isolation.

We rarely have merge conflicts these days, and we are also on 1 big monorepo. It mostly boils down to discipline and architecture.

Re:A Better Model for Stacked Pull Requests by delsarto on Friday April 23, 2021 @08:15PM (#26920815)
This is pretty much done with https://zuul-ci.org/

You simply create a change and in the footer say

Depends-On: <pull request URL/gerrit URL/gitlab URL/etc>

Zuul will pull everything together, merge it all, and provide your tests with repos on disk that reflect the entire dependency chain. You write your test to install/run/etc. from these repos that have been prepared for you. The test might have only your change, or it might have 20 other changes; Zuul handles all this. This way you test everything together.

You don't need to spend resources speculatively running every possible combination; most changes probably don't affect each other (when you know they do; setup dependencies). But the trick is, you don't commit, Zuul does. It puts all approved changes in a queue, merges them and runs the "gate" tests. It does this in a smart way to try and batch as much as possible. When they pass, it commits the change. When something fails; either it won't merge with HEAD, or it's tests fail against HEAD, it gets rejected and you fix it up and go through the cycle again. It's impossible to commit anything broken to HEAD, because everything that is committed has been tested.

Re:A Better Model for Stacked Pull Requests by mdriley on Friday April 23, 2021 @07:38PM (#26920559)
You might enjoy Uber’s paper on SubmitQueue, their system to run CI on what they guess will be the sequence of commits that test successfully: https://eng.uber.com/research/keeping-master-green-at-scale/
Re:A Better Model for Stacked Pull Requests by dmcg on Saturday April 24, 2021 @02:18AM (#26922680)
This. Not the speculative builds thing, but the organising your codebase so that you don’t have this problem.

Our code structure needs to support both the functionality required by our customers, and our needs as developers.

Re:A Better Model for Stacked Pull Requests by jsw on Friday April 23, 2021 @11:27PM (#26922000)
Mergify just introduced a “speculative merge queues[1]” feature that does something very similar.

[1] https://blog.mergify.io/announcing-speculative-merge-queues