- There is only one released version in flight at any time

- You don't need to do a lot of validation before release. E.g. if you release on physical media, or need certification, or shoot the program into space and can't patch it.

Basically, it works for web software where you want (or can have) continuous deployment.

For any scenario that doesn't do "continuous deployment", I think having multiple release/maintenance branches is inevitable. In that case, master is the vNext development branch, and each major version will have a branch for stabilization/validation point releases.

* what is being tested by QA/Product Owner/clients etc right now? What do we do when they actually have feedback that needs fixing?

* how to keep your modifications/patches to SomeUpsTreamSoftware manageable?

* How to deal with parallel development by a team of four? Of twelve? Should all work halt until your PR is merged? What if there are conflicts? How to handle longer lived development?

All of these are hardly ever covered by "keep it simple". Turning that "keep it simple" model into a giant mess of rules, guidelines, scripts and whatnots. Basically a very poor version of Git flow.

Yes. Git flow is often overengineered for CD types of workflows. But it does have all the cases and situations covered that you will eventually run into, and that your homebuilt "simple" flow has not.

Edit: to be clear: there are solutions for all my points in your Simple Workflow, certainly. But here's how git-flow helps:

* release-branches are ideal for putting work in front of stakeholders before release. Allowing improvements on the release. Even in a CD workflow, having something that is separated from "mainline" during the release-window is a good idea. A release-branch is just that. having a separate mainline where during that time new releases can be made is smart too: you don't want to be forced to wait for feature X to land before you can roll out some important fix. I.e. release, master and hotfix branches.

* support- branches in addition to feature branches allow easy rebasing when upstream changes and you need time to re-apply your patches and customisations. You can e.g. branch off a "support-3.1.4" off the 3.1.4 release from upstream and apply your feature/awesome-hack-x on that.

* feature branches are ideal for parallel development. Being able to merge into a branch that does not immediately release (separated from mainline) is smart, because it allows multiple team members to merge to and then make a release. i.e. "development" and "feature"

Another one that I haven't seen mentioned yet is when your software has to be deployed to multiple targets, and each needs the same foundation but also some target-specific adjustments. Maybe you're going to run on different OSes and need to use a different UI on each OS. Maybe you're writing firmware for a range of similar products and each product has its own quirks. Maybe you need to change some code on platform X because there's a known compiler bug where the normal code doesn't work. There are plenty of practical circumstances where this sort of thing can happen. You want to control and ideally automate as much of it as you can, for the same reasons as any other deployment process, and more complicated but systematic branching strategies are often useful for that.

If you need to build for multiple targets or have different special versions, than write your build scripts to produce multiple versions. The Linux kernel builds for lots of targets, but there is only one master tree for it.

Your Linux kernel example might be quite apt here. While the authoritative version may have a single master, how many distros maintain their own fork of the kernel and probably numerous other packages, merging in changes from upstream regularly, but only ever making releases of their distro from their own forks where perhaps they have their own bug fixes or other adjustments applied?

I have a horror of this due to having experienced a strategy where the team took a new (subversion) branch every time they started a new project (i.e. were building a new device in the same family). You had multiple branches with divergent development but lots of common functionality. If you wanted to fix a bug common to all the systems you had to fix it 4 times.

I absolutely believe that there are better ways of implementing that approach than they used, but I also am convinced through experience that you should have one single line of development and everything else is just 'views' on that (e.g. point release branch is just a view master that is frozen, plus some filtered recent history).

To take your Linux kernel example - the various forks (I'm working with the Xilinx Petalinux Kernel now) don't really fit into the normal model. They're a combination of a release branch and a feature branch - they take the Linux Kernel at a certain time, then add some patches on top. Those patches then usually get submitted (indirectly) back to Linus so that they join the master path of development. That means that other people benefit from them, and refactor them when appropriate.

Yes. That is exactly what you do.

How do you know which ones are going to break the build, how do you know which ones just aren't going to merge due to divergence?

You use tools like unit testing and continuous integration running on all of your key branches, just like you would on a single-master project.

I have a horror of this due to having experienced a strategy where the team took a new (subversion) branch every time they started a new project

That brings back painful memories. I think it's fair to say that in the area of branch management, modern VCSes are much easier to use than Subversion. Even if you're not using them as physically distributed, the same tools for readily transferring work across branches and tracking what has gone where are also useful in other scenarios such as the one we're talking about here.

I also am convinced through experience that you should have one single line of development

I guess this is just a case of everyone's experience being different. My experience has been that projects that try to keep everything on a single branch when the reality is that you will be making several significantly different types of build over the long term can become prohibitively expensive to manage. You don't necessarily have to use a multiple branch strategy to deal with that, or that strategy alone; I mentioned a couple of other useful options earlier on. But trying to use a single master and then make multiple types of build regularly using just informal processes or ad-hoc changes on release branches is a recipe for human error IME. Something has to systematically track and control the divergence or things get really messy.

They're a combination of a release branch and a feature branch - they take the Linux Kernel at a certain time, then add some patches on top. Those patches then usually get submitted (indirectly) back to Linus so that they join the master path of development.

This is exactly what typically happens in the situation I was describing as well.

You would presumably have some root with the common code, and then you maintain your per-target branches off that. Most development work still gets pushed to the common root as usual, and then all targets benefit unless they're specifically overriding the default code in that area (which is handled at the point of merging into the target-specific branch).

Any changes first made on target-specific branches can also be passed up and then back down onto other targets if it turns out that they are more widely useful, or transferred directly to one or more of the other target-specific branches but without applying the change to the common root. However, typically you would do anything like this via some sort of cherry-picking process; you would never normally transfer all of your changes back up by merging a target-specific branch back into the shared root.

There are better ways to manage that.

I'm following the process they described on a team of about 40 developers, and it actually greatly reduces those problems by forcing people to simply not to have long lived branches. Git flow just puts off the pain of working with that many people until later and creates a giant train-wreck when you finally try to incorporate your feature branch.

If you have a large feature, then use feature flags to let you release it to production dark before it finishes. If you want to do a large refactor, either find a way to do it incrementally or keep both the old and new versions in place together until the new version is completely done.

> release-branches are ideal for putting work in front of stakeholders before release

Feature flags are better for this. If you don't want users to see an incomplete feature, that doesn't mean they have to be using a version of the software that doesn't support that feature. You can just explicitly hide the feature from certain classes of users.

Other criticisms of git flow and alternative branching procedures have come up over the years [2] and been discussed here [3][4][5].

A good, simple metric for whether your team is using git well or not is how often a developer is forced to play Guitar Hero [6] just to fix up a bad merge or rebase. The ideal answer is "never"; if it's happening more than a couple times a year, something's wrong.

[1]: https://gist.github.com/jbenet/ee6c9ac48068889b0912

[2]: https://barro.github.io/2016/02/a-succesful-git-branching-mo...

[3]: https://news.ycombinator.com/item?id=9744059

[4]: https://news.ycombinator.com/item?id=22485489

[5]: https://news.ycombinator.com/item?id=11190310

[6]: https://twitter.com/HenryHoffman/status/694184106440200192

Git flow is something I spent a lot of time theory crafting about years ago. I’d say wasted time, but learning is never a waste and one of the things I learned is the value of deferring complexity whenever possible, especially if there’s no downside.

Using a trunk and tag (that’s what I call it) workflow today doesn’t prevent you from adopting something more complex if the need arises, so it makes a ton of sense for it to be the default on new projects, especially for new developers IMO.

This flow works under some conditions only:

* You have working CI/CD that you can trust

* You do not have to support old versions of the software

Example of some situations where these conditions are not met:

* Installs of your software are not reachable over the Internet

* Your software needs manual testing or qualification before deployment

* You do not handle your software's deployment; your client does.

In those situations, you may need to maintain release branches, and backport fixes, etc.

- This flow applies more for a service or a website, where there is no problem of distribution: the distribution is done the moment your user reaches your website/service.

- In the case of a website there is one website. In the case of services there may be multiple versions, but they exist together in the same file tree. Fixing an issue on an old version means fixing the issue on specific files; when they're pushed on master the users will get the fix

- An alternative flow of having to manage multiple versions in parallel is to use feature-flags to compartmentalize sections of code that aren't tested yet or not even functional yet. That way you can have everything on a single branch and make all the code aware of each other. Feature flags can be activated gradually in qa environment, then in staging, then in some customers only, etc... to iron out the bugs as deployment goes. Note that this way of doing is also adapted for traditional installable software, see Firefox and Chrome and their myriads of activation flags.

The advantages of feature flags over feature branches are mostly an illusion, though.

Sure, if you're updating your code in co-ordination with updating some external system, such as a database migration, then sometimes deploying in stages with multiple versions of some functionality running concurrently during some of the intermediate stages is more-or-less the only way you can do it without incurring downtime.

But otherwise, even if you have 10 teams and each hides their changes behind a flag and pushes to master instead of a feature branch, you still have an exponentially growing number of combinations and they're still not all being tested in arbitrary combinations. Turning the feature flag on is then little different to merging in a feature branch: it's still a time when things can go wrong. It's just that now they're going wrong on your master branch and screwing things up for everyone else, too.

So you're essentially advocating testing new code, or new combinations of code changes, in production?

For a feature flag to be useful compared to the typical setup where you have a feature branch and then rebase that onto master from time to time until you're ready to do a fast-forward merge, you would need to be testing multiple features concurrently before any of them is merged back to master.

Testing all possible combinations of unmerged features in development is an exponentially difficult problem, so surely no-one is doing that. However, even testing selected combinations still requires someone to be aware of everything that is going on in enough technical detail to highlight potential trouble areas in time to take useful action that would become more difficult once some of those features start to get merged in.

Assuming you have a large enough project that co-ordinating different ongoing development work isn't just handled via informal discussions anyway, that would mean someone has to be identifying risky combinations that could benefit from testing in advance, and someone has to be responsible for doing that testing, and someone has to be responsible for addressing any issues that are discovered as a result. Does anyone here work in a development group that actually does this?

Meanwhile, the cost of using feature flags routinely is clear: every time you do any significant development work, you have to bracket all affected code with a new flag, and someone has to coordinate those flags and make sure they're all tidied up with code left in the "on" mode by release time.

The thing that really helped us was separating the idea of environments and branches.

If master can be unstable then you can't seamlessly release and must test.

If you need to keep master stable then it can't be easily merged into by developers. (testing a tag is not enough because iterations with have other changes in the mean time.)

People attempt to skip these issues by embracing the chaos with CI/CD but that's outside the scope of the branching strategy. Maybe the simple flow is the best but it has clear stumbling blocks you will hit.

The question is where do you want to discover them? You either run tests against the feature branch before merging or you have to gate your master branch into a version on a test system before it is promoted to production.

CI/CD can be chaos but the Facebook philosophy is that we would rather you delivered 1000 features per day and be able to quickly fix anything you might break in the process rather than release 1 thing a day that is fully tested. Might not work for your system but it makes sense to them.

Or you can cut a release branch, test, and iterate on that branch and be secure knowing only those release iterations are being made.

Its fine for master to have bugs if you're on a release branch. That's why people use them.

> Each issue in GitHub is 1:1 with a PR that resolves it.

> All normal work is branched directly off latest master.

> Every branch is rebased against master prior to merging.

> Feature flags are heavily used to gate the activation of functionality which is not applicable, not ready, or otherwise in need of being configured per environment+client.

> Checkbuilds complete with success before we merge to master. Nothing is allowed to merge which could impact production without a feature flag or other gatekeeping mechanism in place. We have a labeling system to track which issue+client pair needs a feature flag configured in a certain way so this can be managed at deployment time.

> Hotfixes are managed by branching off the commit deployed to the client+environment of concern. We have a system (in house developed) that tracks all of the current client+environment+hash tuples so it is very easy to know where to stick the shovel every time.

> We effectively consider the commit hash to be the version of our software. Our releases are extremely granular due to the feature set we support for our customers, so commit hash is the best way to manage this. That said, it is not unusual for multiple client+environment pairs to share a common commit hash depending on our release timing (this is purely asynchronous and managed by our project group).

That's about it. There is only 1 permanent master branch. No work branch ever sits longer than ~2 weeks in our development model. We merge code ASAP so that we can iterate on the results.

For reference, we merge to master ~10-30 times per day. I have some plans in the pipeline for implementing a technique that would allow for pushing hundreds or thousands of commits per day if we ever got to that point.

Another problem my organization has hit with this approach is that, unless you somehow maintain 0 bugs level, you get to a point where you need to do 2 separate things: fix bugs on features that are already "done", while still developing new features, while integrating those in rapid succession. This is mostly true when preparing for a release while still having to work on features for future releases.

This can lead to the need to have two separate "main" branches, one that is used for stabilization, and another that is used for active development.

Merge to master, short-lived feature branches, and when it's release time, git checkout -b v1.5, git tag -a 1.5.0, and then cherry pick bugfixes onto that branch when it's time for 1.5.1. If you need more time to stabilize for 1.5.0, just add some time & development between cutting the branch and tagging it.

The real time processing runs on stable releases which are tags created off of a stable branch (as you have suggested) and all the other processing runs off the tip of master.

I don't think the term "overengineering" applies when you take an industry practice off the shelf. If anything is overengineered, it would be the simple solution that you "engineer" in house.

I think that is quite a good description of a common problem in software development these days.

Needless to say that I have been guilty of indulging in it

Consider also that sometimes complexity/engineering really is warranted; either by the domain or by the tooling.

In other words, avoiding complexity-domain mismatch isn't just a "these days" problem, it's a constant consideration in software development.

[0] https://trunkbaseddevelopment.com/

Just having one release branch per version was still generating too many errors, so now we have one per version per environment.

This would work much better if the company would get it’s head around continuous deployment, but they just don’t want to release anything without a full round of manual regression testing.

At first I think people doubted that it could be so simple. It's like they'd been forced to learn something incredibly complex and just assumed they weren't clever enough to know any better. A kind of Stockholm syndrome maybe? We've been running the simplified system for more than a year now and everyone is happy. Not once have we ever needed anything even remotely like git flow again.

Git-flow is the answer in most of those scenarios and the only way to keep your sanity.

What you describe is the happy path scenario.

However, in practice you'll have to deal with legacy software (that doesn't even have to be that legacy), with colleagues that aren't aligned with it, and with people being afraid.

Currently I'm working in a company that mainly does nightly builds and manual testing, but I'm working in a new project and trying to make it work like you say. I am constrained though, they have Jenkins as CI which is a bit painful to work with (as in it'll take me a lot of work to get things where I want them to be). Working with pre-commit, pre-push hooks and a solid IDE goes a long way though.

You need to be able to backport fixes for that customer who pays your bills, but is 6 versions behind.

That said though, software should be built to allow for fast and painless upgrades. Backwards compatibility and many use cases should be tested automatically and constantly. But, it's a big investment to have software like that, and you need to resist a lot of younger, eager developers that want to e.g. introduce a new language or make sweeping changes.

A bugfix to the integration is typically low risk and high impact, and so the client will want to install that ASAP. For example it could be they suddenly changed the values in a single field in an XML file we read, due to upgrades elsewhere in the organization yay, halting the integration.

So we push a fix where we handle the new values, and they want that installed ASAP.

What they do not want pushed out into prod without lots of testing is all the new things we've added elsewhere in our product, or larger changes to existing features.

What the GP suggested I assume is one master-like branch for every deployed version. Otherwise it makes no sense.

If you don't use the optional parts of Gitlab Flow like a production branch, what you are left with is feature branches merged to master, and then release tags made from master. The only difference is that both of us don't make the release branch unless it's necessary, we just make the tag.

This is what we do at work.

Unless there is some major database or library or language version change in a feature branch, once that branch is tested and QA’d, the M.O. becomes “merge any feature branches that we’re ready to put live into master and release”.

Long-lived branches are a mess, and rebase flow is crushed by them.

If you can fit this kind of simple Git workflow into your development workflow, I recommend it.