Re:Patterns for Managing Source Code Branches
by BerislavLopac on Friday May 01, 2020 @05:16PM
(#23046585)
I have been practising something that we might call "continuous rebase", for the lack of better term. It is essentially a combination of feature branches and continuous integration, but instead of making frequent merges from a feature branch into the master it is done in the opposite direction: the feature branch is frequently rebased on top of the master (or, at the very least, the master is merged into the feature branch, if the rebase is too difficult).
This approach has the advantages of both of the other approaches:
* the code is integrated with a great frequency
* any conflicts can be resolved in the feature branch and will not break the master
* the new features are isolated, and there is no need for feature switches
Re:Patterns for Managing Source Code Branches
by _fourzerofour on Monday April 20, 2020 @11:04PM
(#22931419)
This is fantastic, having looked through Martin's blog before and found some gems that really leveled up my process on managing code.
As an aside - beyond the coming installments of this series, can anyone recommend sources for setting up CI locally, or for small teams? DDG is giving me pages on GitHub Actions, Atlassian, and DigitalOcean blogs, but these are all process or product specific.
Re:Patterns for Managing Source Code Branches
by stevekemp on Tuesday April 21, 2020 @04:44AM
(#22933160)
Mostly the driver doesn't matter; for the CI stuff you basically want to carry out "actions" on "events". As a concrete example you might wish to run your test cases when a new pull-request is opened, or an existing one is updated.
The way that glue/integration works is going to be somewhat specific to the setup you're already using. That's why you'll find Github Actions being documented for Github, Jenkins for self-hosted repositories, etc.
Broadly speaking it doesn't matter which "thing" you use to drive your CI, the key is that it works reliably and that the stuff which is executed is inside your repository. That way:
* Developers can run it manually if they wish (e.g. "make test")
* The CI system can be swapped out in the future, if you need to.
As a concrete example once I migrated a company's repositories from bitbucket, using their pipelines, to (self-hosted) Github Enterprise. Because the pipeline just said "Run .git/tests.sh" porting the glue was trivial.
I even published a github action for those using github, so that you can launch your project-specific tests in a portable and simple fashion:
Re:Patterns for Managing Source Code Branches
by noxToken on Tuesday April 21, 2020 @12:01AM
(#22931721)
CI will always be product specific unless you're going to create your own scripts for your pipeline. Even then, your script for executing tests, deploying, etc. will still be dependent on your stack.
Some general guidelines:
When a developer's branch is pushed publicly, compile it and run tests. It must pass before being allowed to merge into develop/master/${specific-branch}. Require that merges to those branches are only fast-forward merges.
The types of tests needed will be dependent on your application. Make sure they're all green before proceeding. If not, log the failing tests. Depending on your application, you may want to have certain testing metrics also fail the build. For example, require that at least 50% of new lines of code have test coverage and 60% of your overall source code have coverage.
Run any static analysis tools and linters. Configure which rules should trigger failures or warnings.
Run any other scripts that aren't related to code quality like minification or building binaries.
Deploy your application to the proper environment.
This is a bare-bones list. Your application may have many more steps, but this should get you started.
Re:Patterns for Managing Source Code Branches
by PhilippGille on Tuesday April 21, 2020 @02:46AM
(#22932603)
Are you looking for advice for tools or for how to configure a pipeline?
The other answer mentions some pipeline steps (lint, test, compile etc), so I'll focus on tools.
The heavyweight champion for self hosted CI is still Jenkins I think. Pipeline config files are written with a scripting language (Groovy) instead of yaml, which gives you a lot of options for customization: https://jenkins.io/
But for a first CI setup for a small team it might be too much. I think Drone is lighter and simpler, while still being popular enough for having a community that creates many plugins: https://drone.io/
Re:Patterns for Managing Source Code Branches
by corpMaverick on Monday April 20, 2020 @07:46PM
(#22930230)
Nice. I liked Ken Beck's quote. Insightful as always.
As far as I remember, Continuous Integration (CI) was about merging your code with the rest of the team as frequently as possible. Jenkins, and other tools were not even in the horizon. Because in the waterfall prehistory, some teams used to divide the system in parts and integrate the source code at the end. And it was hell. So now we have these powerful tools like git, and in a way we use them to cheat and not do CI.
Re:Patterns for Managing Source Code Branches
by NomDePlum on Monday April 20, 2020 @10:45PM
(#22931314)
Nothing to stop code reviews either. Just don't make them a gateway. Allow code to be merged then reviewed if necessary. If you have good common coding practices and testing then likely more than safe.
Re:Patterns for Managing Source Code Branches
by corpMaverick on Wednesday April 22, 2020 @11:59AM
(#22946961)
Great advice. This is why it is good to go back to the basics. Adding gateways goes against any lean principles, introduces delays (=waist) in the process.
This approach has the advantages of both of the other approaches:
"Streamed Lines: Branching Patterns for Parallel Software Development" by Brad Appleton, Stephen P. Berczuk, Ralph Cabrera, Robert Orenstein. [1]
It was presented at Pattern Languages of Programs '98 (PLoP '98).
Incidentally, looking at the proceedings [2], Martin Fowler as co-author on another paper at the same conference called "Temporal Patterns".
[1] https://hillside.net/plop/plop98/final_submissions/P37.pdf
[2] https://hillside.net/plop/plop98/final_submissions/
As an aside - beyond the coming installments of this series, can anyone recommend sources for setting up CI locally, or for small teams? DDG is giving me pages on GitHub Actions, Atlassian, and DigitalOcean blogs, but these are all process or product specific.
The way that glue/integration works is going to be somewhat specific to the setup you're already using. That's why you'll find Github Actions being documented for Github, Jenkins for self-hosted repositories, etc.
Broadly speaking it doesn't matter which "thing" you use to drive your CI, the key is that it works reliably and that the stuff which is executed is inside your repository. That way:
* Developers can run it manually if they wish (e.g. "make test")
* The CI system can be swapped out in the future, if you need to.
As a concrete example once I migrated a company's repositories from bitbucket, using their pipelines, to (self-hosted) Github Enterprise. Because the pipeline just said "Run .git/tests.sh" porting the glue was trivial.
I even published a github action for those using github, so that you can launch your project-specific tests in a portable and simple fashion:
https://github.com/skx/github-action-tester
Some general guidelines:
When a developer's branch is pushed publicly, compile it and run tests. It must pass before being allowed to merge into develop/master/${specific-branch}. Require that merges to those branches are only fast-forward merges.
The types of tests needed will be dependent on your application. Make sure they're all green before proceeding. If not, log the failing tests. Depending on your application, you may want to have certain testing metrics also fail the build. For example, require that at least 50% of new lines of code have test coverage and 60% of your overall source code have coverage.
Run any static analysis tools and linters. Configure which rules should trigger failures or warnings.
Run any other scripts that aren't related to code quality like minification or building binaries.
Deploy your application to the proper environment.
This is a bare-bones list. Your application may have many more steps, but this should get you started.
The other answer mentions some pipeline steps (lint, test, compile etc), so I'll focus on tools.
The heavyweight champion for self hosted CI is still Jenkins I think. Pipeline config files are written with a scripting language (Groovy) instead of yaml, which gives you a lot of options for customization: https://jenkins.io/
But for a first CI setup for a small team it might be too much. I think Drone is lighter and simpler, while still being popular enough for having a community that creates many plugins: https://drone.io/
As far as I remember, Continuous Integration (CI) was about merging your code with the rest of the team as frequently as possible. Jenkins, and other tools were not even in the horizon. Because in the waterfall prehistory, some teams used to divide the system in parts and integrate the source code at the end. And it was hell. So now we have these powerful tools like git, and in a way we use them to cheat and not do CI.
I think I’d rather have CI.