News for hackers. Stuff that matters.
Re:All of us test in production all the time (2019) by UweSchmidt on Sunday July 19, 2020 @04:50PM (#23892326)
As QA I think testing, test environments, test ressources should be first class concepts in software. Building whatever "modern" architecture twice, and simulating traffic is not easy but possible, at least to an extent. You need to get that extra license for any piece of software for testing during procurement, and provide means to create test ressources, like typically test users as needed. Sounds trivial, but is often so complicated (banks, insurances...) that it should have been considered early during design.

So, I wouldn't use the word "lazy", but we could do better.

Re:All of us test in production all the time (2019) by virtue3 on Sunday July 19, 2020 @11:42PM (#23894479)
It's not the architecture, it's the representing all the states in prod in testing.

Not saying it's impossible, just not always feasible to have it actively "perfect".

I've found that an "ok" testing envrionment and solid unit testing / monitoring tends to do better.

And then b/g deploys and canaries :/

Re:All of us test in production all the time (2019) by sagichmal on Monday July 20, 2020 @08:47AM (#23897200)
Modern architectures look like this: https://github.com/donnemartin/system-design-primer

Building this out twice is either actually impossible, or so cost prohibitive as to be practically infeasible.

Re:All of us test in production all the time (2019) by eitland on Monday July 20, 2020 @10:04AM (#23897956)
> Modern architectures look like this:

A lot of modern architecture is not like that.

And even the diagram there is something should be possible to replicate if the organization values it.

Today it doesn't even need to be too expensive as it can be deployed with terraform and torn down an hour later when the full system tests are finished qnd go back to running the integration tests.

Re:All of us test in production all the time (2019) by sagichmal on Monday July 20, 2020 @11:39AM (#23898868)
It's not possible to replicate the CDN, or DNS, or the message queue if it's hosted, or the database of it's big enough, or etc. etc. The differences between what you're able to create in a staging environment and what exists in prod are significant enough that testing against the staging environment doesn't build much more confidence than testing with mocks.

Testing in prod means accepting this reality, avoiding the largely unproductive toil of building and maintaining a staging env, and making the production system observable, resilient, and operationally agile enough that you can deploy changes that can trigger unpredictable and emergent system behaviors while managing risk appropriately.