We don’t use a staging environment

Re:We don’t use a staging environment by Gigachad on Sunday April 03, 2022 @05:37PM (#30900928)

This is how my current place does it. The only issue we are having is library / dependency updates have a tendency to work perfectly fine locally and then fail in production due to either some minor difference in environment or scale.

It's a problem to the point that we have 5 year old ruby gems which have no listed breaking changes because no one is brave enough to bump them. I had a go at it and caused a major production incident because the datadog gem decided to kill Kubernetes with too many processes.

[ Parent | on HN ]

Re:We don’t use a staging environment by ehnto on Sunday April 03, 2022 @10:11PM (#30902727)

Do you have a replicate of the production environment codified somehow, like into a VM? It's rarely perfect, but I usually try and develop locally on the same stack I deploy to which can help with the environment differences.

It's also why I think it's smart to rebuild the environment on deploy if it makes sense for your pipeline, so that you wipe any minor differences that have been accruing over time. Working on a long running product, you quickly find yourself with disparities building up, and they're not codified, so they're essentially unknown to the team until they cause an issue.

[ Parent | on HN ]

Re:We don’t use a staging environment by Gigachad on Sunday April 03, 2022 @10:51PM (#30902930)

Yes, there is a docker compose file which is usually sufficient. The issue in this case was the problem only shows up if there is sufficient load on the background workers.

[ Parent | on HN ]