News for hackers. Stuff that matters.
Re:Google Cloud Incident Report – 2025-06-13 by usrnm on Saturday June 14, 2025 @10:26AM (#44276647)
How is the fact that it was a database change and not a binary or a config supposed to make it ok? A change is a change, global changes that go everywhere at once are a recipe for disaster, it doesn't matter what kind of changes we're talking about. This is a second Crowdsrike
Re:Google Cloud Incident Report – 2025-06-13 by fidotron on Saturday June 14, 2025 @10:41AM (#44276726)
This is the core point. A canary deployment that was not preceded by deploying data that activates the region of the binary in question will prove nothing useful at all, while promoting a false sense of security.
Re:Google Cloud Incident Report – 2025-06-13 by ethbr1 on Saturday June 14, 2025 @01:22PM (#44277565)
The root problem is that a dev team didn't appropriately communicate criteria for testing their new feature.

Which definitely seems like shortcut 'on to the next thing and ignore QA due diligence.'

Re:Google Cloud Incident Report – 2025-06-13 by sagarm on Saturday June 14, 2025 @01:43PM (#44277680)
Google does not separate dev and QA in general (I'm sure there are exceptions).
Re:Google Cloud Incident Report – 2025-06-13 by ethbr1 on Sunday June 15, 2025 @07:50PM (#44285709)
That sounds like a conflict of interest in practice.

Who's ass is responsible when a dev team starts missing coverage in their tests, like happened here?

Re:Google Cloud Incident Report – 2025-06-13 by sagarm on Sunday June 15, 2025 @11:30PM (#44286423)
...the engineering team? I'm confused by this idea that engineering teams are not responsible for software quality. Who else would be?

My experience with separate QA teams was not encouraging. They dramatically slowed down releases while failing to catch bugs. Their capability to automate was poor, so they were doing manual testing that provided little coverage especially compared to our comprehensive submit-blocking unit and integration tests.