> After the end of the system-assigned 1 year period, the customer’s GCVE Private Cloud was deleted. No customer notification was sent because the deletion was triggered as a result of a parameter being left blank by Google operators using the internal tool, and not due a customer deletion request. Any customer-initiated deletion would have been preceded by a notification to the customer.
... Tada! We're so incompetent we let giant deletes happen with no human review. Thank god this customer didn't trust us and kept off-gcp backups or they'd be completely screwed.
> There has not been an incident of this nature within Google Cloud prior to this instance. It is not a systemic issue.
Translated to English: oh god, every aws and Azure salesperson has sent 3 emails to all their prospects citing our utter fuckup.
Except that, from the article, the customer's backups that were used to recover were in GCP, and in the same region.
https://www.unisuper.com.au/contact-us/outage-update says "UniSuper had backups in place with an additional service provider. These backups have minimised data loss, and significantly improved the ability of UniSuper and Google Cloud to complete the restoration."
No doubt this little bit must be causing some annoyance among UniSuper's tech teams.
“Data backups that were stored in Google Cloud Storage in the same region were not impacted by the deletion, and, along with third party backup software, were instrumental in aiding the rapid restoration.”
Where exactly do they mention they have confirmed soft delete behavior systemically? All they said was they have ensured that this specific automatic deletion scenario can no longer happen, and it seems the main reason is because "these deployments are now automated". They were automated before, now they are even more automated. That does zero to assure me that their deletion mechanisms are consistently safe, only that there's no operator at the wheel any more.
They could have gone one step further by reviewing all cases of default behavior for anything that might be surprising. That said, it can be difficult to assess what is "surprising", as it's often the people who know the least about a tool/API who also utilize its defaults.