Question 1: What's running in production? Allowing production to run "anything" means "anything" could be running there, so you need tooling to answer that question. Maybe that's just "give every developer read-level access to AWS", or maybe its a slack slash command, or something, but there's a cost there. With a coordinated model: its just the main branch. Its always the main branch. So look at the main branch.
(Sidenote: this isn't a perfect solution, because oftentimes there's a temporal delay between a main branch merge and deployment, and developers want to know precisely when the cutover starts/finishes. So you end up building that extra stuff anyway. But: that can be delayed quite a bit, as its a higher level question given that the main-is-production model solves a lot of problems alone).
Question 2: How do I re-run the build? Well, you can push a new commit, or issue some slack slash command, or maybe a comment in the PR that reads `!buildbot run` or whatever. How about we just make it simpler? Just push a new commit.
The broader point is that: turing machines can do anything. Human brains can't. So when operations are simplified around models like "source-is-truth" its because that model is a satisfying fit for the problem space which helps the human brain reason about what is happening. All models are wrong; but some are useful.
You can setup your deployment tools to always deploy the lastest from a specific branch unless manually overridden. It is also not very difficult to post the deployed got commit hash to an internal channel, or even expose it via a web interface. Node of this require designing your system so that the only way to trigger a new deployment is by pushing a new commit.
> Well, you can push a new commit, or issue some slack slash command, or maybe a comment in the PR that reads `!buildbot run` or whatever. How about we just make it simpler? Just push a new commit.
And what happens if you want to test building before your change is fully ready to merge or roll back to an older release? It is better to design a modular system than to try to hack this basic stuff in when you suddenly need it.
Then there's my security concerns of conflating "write access to github" full access to the environment. If you allow code to be deployed immediately on commit, then anyone who can commit can compromise your environment.
VCS can only ever be intent at best.
Velocity is relative in physics, right? Relative to what? What's the frame of reference? Usually, we say, the Earth itself. VCS is that to software engineering, in this setup; there's movement and realities outside of VCS, but VCS is our zero; the tare. Everything is relative to the VCS. It is not truth through its nature; it is truth through how we treat it, how we build systems to interface with it, synchronize with it, etc.
All models are wrong, but some are useful.
I've uh, not seen one that doesn't? (I agree with you, it shouldn't. Does your CI system not have a "retry" or "restart" button?)
> The most shocking of all, though, and I have actually seen this: If you deploy a broken build, a rollback to an earlier version should never, ever, require a rollback on your source control.
So, my company does this, and I love it; it gives even the least skilled of our devs a good shot at the "recognize things aren't working → problem started with last deploy → revert last deploy" debug path, and getting it right. Having that last step of revert a deploy literally correspond to a git revert is very useful & easy.
Keeping the "state of what is deployed" in code also gives us good confidence that there aren't random changes being made, allows for code review, provides a log of who changed what when — all the goodness of infrastructure as code.
Now, we keep the main production "monorepo" separate from our repo that describes what is deployed (let's call it "deploy-state". So a revert of a deployment happens in the "deploy-state" repo, not in the monorepo. So, someone's merged feature branch isn't going to get reverted. (Though, we should have a conversation about whatever the actual problem is: if it is code in the monorepo, why did it not fail tests there? We also have ways of saying "this range of commits, from [break, fix), should not get deployed to prevent repeats.)
Whether that setup is a "true monorepo" or not, since it literally involves more than one repo … well, you be the judge. It seemed to us pragmatic to have it be separate, and we've not regretted that.
This makes me think you're disagreeing with the literal interpretation of what I wrote, while very much agreeing in spirit.
I'm not trying to say that a VCS is an intrinsically wrong solution to managing deployment state (I'm not a fan, but it's an eminently reasonable approach). My issue is with deployment state and application source being enmeshed in the same repo such that you can't evolve one without messing with the other.
It's pretty common on GitHub that PRs trigger a build but unless the author has some permission on the target project already, can't trigger a rebuild. I'd estimate 30%+ of the PRs I create fail for reasons unrelated to my changes and I have to ask someone to retry.
I have never seen any of the above issues.
Why not? In some cases in lighter weight systems, you might have a release branch that gets pushed to production. Like dokku.
The other side of the equation is that if I break something, I want to make the smallest change possible that fixes it, so as to avoid confounding variables if it doesn't work. Off the top of my head:
* I don't want to make things worse by messing up the rollback due to stress
* I don't want to trigger a build and wait for CI to run and build
* I don't want to depend on my build not being perfectly reproducible (e.g. because the problem comes from a bad dependency upgrade and the rollback rebuilds with the bad dependency)
I hate Atlassian Bamboo with a passion but it is the only CI/CD system I remember coming across that actually has a concept of deploying a built artifact.
If the build fails, you should be able to re-run it without pushing to version control. You can e.g. have builds fail due to issues with your artifact repository that don't require any changes to the code to debug/fix. You can have issues with deployment after the build completes, you don't want to have to rebuild to redeploy. The most shocking of all, though, and I have actually seen this: If you deploy a broken build, a rollback to an earlier version should never, ever, require a rollback on your source control.
As always, it's very much a case of too much of a good idea. People correctly identify that automation is a good thing, but then turn that into "manual operation is a bad thing, and therefore we shouldn't cater to it".