Re:Knightmare: A DevOps Cautionary Tale (2014)
by hcarvalhoalves on Wednesday February 04, 2015 @08:32AM
(#8996675)
> I don't get that. There was no code issue. The old and new code both worked as intended, it was a deployment and deployment-verification problem.
A code review could raise the issue of repurposing a flag in case they had to revert the deploy. Changing the semantics of a flag is a big no-no anyway, and there are ways to guard against that.
> Yes they did. They staged the new code and tested it. They did a slow deployment also.
But they didn't had a staging environment that matched their live environment, apparently. You want a staging environment that is 1:1.
> They did revert the deploy within the 45 minutes. It made it worse.
If you think reverting a deploy by simply pushing an older version is the same as a contingency plan, think again.
A code review could raise the issue of repurposing a flag in case they had to revert the deploy. Changing the semantics of a flag is a big no-no anyway, and there are ways to guard against that.
> Yes they did. They staged the new code and tested it. They did a slow deployment also.
But they didn't had a staging environment that matched their live environment, apparently. You want a staging environment that is 1:1.
> They did revert the deploy within the 45 minutes. It made it worse.
If you think reverting a deploy by simply pushing an older version is the same as a contingency plan, think again.