News for hackers. Stuff that matters.
Re:Shipping is your company’s heartbeat (2013) by BillinghamJ on Sunday September 24, 2017 @12:30PM (#15325355)
You deploy gradually, initially to 1% or so of requests. If your metrics, KPIs, etc. don’t suffer, it continues to roll-out until 100% of requests are using that code-path - within a few hours or so.

Takes a bit of infra effort, but for a company like Intercom, that’s very easily achievable.

Re:Shipping is your company’s heartbeat (2013) by AznHisoka on Sunday September 24, 2017 @02:16PM (#15325795)
this is easy and doable if your change is a front end change.

good luck trying this when its a backend change involving distributed threads, locks, resources, etc where production load is 100x more than staging/test.

Re:Shipping is your company’s heartbeat (2013) by greenleafjacob on Sunday September 24, 2017 @02:49PM (#15325923)
If the cost of downtime is high enough you can mirror some or all traffic onto a shadow copy of the infrastructure.
Re:Shipping is your company’s heartbeat (2013) by BillinghamJ on Sunday September 24, 2017 @03:04PM (#15325984)
Obviously in those cases, you’d take more than a minute to review prior to deployment...

There are plenty of backend changes where your list of complexities do not apply.

Re:Shipping is your company’s heartbeat (2013) by ben_jones on Sunday September 24, 2017 @12:49PM (#15325442)
what would be some good resources to learn how to build such infrastructure? I've seen it as a feature of various platforms but would imagine it could become complex otherwise.
Re:Shipping is your company’s heartbeat (2013) by BillinghamJ on Sunday September 24, 2017 @02:06PM (#15325769)
It wouldn’t be practical without some kind of underlying infrastructure orchestration system. Obviously some companies have gone the extra step to actually build these systems - e.g. Google made Kubernetes. For any not-huge companies, I think it pretty much just falls to choosing one of the many existing tools.

Obviously it is possible to just throw together a bunch of scripts and vaguely make it all happen, I’m not sure I’d be comfortable to do that in production.

Re:Shipping is your company’s heartbeat (2013) by toast0 on Sunday September 24, 2017 @06:02PM (#15326720)
The simple way to do this can apply if you're mostly doing things in the context of a user. Write the change to check if a hash of the user identifier is less than a target; increase the target over time.