News for hackers. Stuff that matters.
Re:Inside the longest Atlassian outage by drsim on Wednesday April 13, 2022 @05:16PM (#31020394)
Which SaaS platforms provide account-level restores?

If you contact them and say "please restore our data to as it was last week" those I know do not offer this.

Re:Inside the longest Atlassian outage by ibejoeb on Wednesday April 13, 2022 @05:50PM (#31020739)
I did. It was an first-principles architectural decision. A client could request any point-in-time within the contracted period, and it could be either a restoration or a fully operational, parallel instance of the account.

It was initially a cover-my-own-ass design, but it turned out to be an extremely popular feature that was never even used for disaster recovery. Instead, it was used for audit support, trial scenarios, projections, and all kinds of other stuff.

Re:Inside the longest Atlassian outage by jimbokun on Wednesday April 13, 2022 @10:10PM (#31022689)
What kind of data volumes were you dealing with?
Re:Inside the longest Atlassian outage by boardwaalk on Wednesday April 13, 2022 @05:30PM (#31020552)
I wouldn’t expect them to advertise such a thing, but the question is “can they recover from their own mistakes” not “can they recover from mine.” I don’t care if this is with an “account-level restore” or whatever; it shouldn’t be my concern.
Re:Inside the longest Atlassian outage by brazzledazzle on Wednesday April 13, 2022 @09:18PM (#31022349)
I’ve seen customer and resource level restores deprioritized more than once and the only hypothetical given serious thought is avoiding helping customers who accidentally deleted something because of the support burden/cost. No one seems to have much concern for what happens when they’re the ones that screwed up.
Re:Inside the longest Atlassian outage by twistedpair on Wednesday April 13, 2022 @07:51PM (#31021772)
I know plenty of places (small/med startups) with "undelete" and "restore" account/data options built into their admin panels. Engineers shouldn't be flipping bits by hand, under duress.

I really wonder what these Altassian restore tools look like it if takes "hundreds of engineers across the company" to restore 400 accounts. Are backups siloed across many teams?

Re:Inside the longest Atlassian outage by kgeist on Wednesday April 13, 2022 @07:17PM (#31021501)
>Which SaaS platforms provide account-level restores?

We restore deleted accounts on request sometimes. There was a client, for example, who forgot to renew the subscription and did nothing for 30 days, so their account was automatically deleted. We restored it from backups. It helps that every tenant has their own isolated database, so it's mostly a matter of restoring that one single DB. Some microservices store data without DB-level sharding, so we have a script which is able to make a partial dump for a specific account.

There's also a popular option to restore deleted data - nothing is ever hard-deleted (it's marked deleted but stays in the DB) and we have a script which can restore individual records (and related records). There's maybe 5 such requests per month.

We don't offer rolling everything back to a specific point in time, though. Technically it's possible by undoing the event queue but it's untested.

We also have a script to migrate customers from cloud to on-premises and back.

Re:Inside the longest Atlassian outage by Tobani on Wednesday April 13, 2022 @06:02PM (#31020849)
I accidentally built out this feature at a company once and it totally saved our asses a week later.
Re:Inside the longest Atlassian outage by hotpotamus on Wednesday April 13, 2022 @05:22PM (#31020450)
I actually did this once with Dropbox, though it wasn't a feature they actually published. I clobbered my Dropbox directory accidentally, but I was able to find a script someone wrote to roll it back to a previous point in time and it worked quite well. After that I also took my own snapshots just in case.
Re:Inside the longest Atlassian outage by MapleWalnut on Wednesday April 13, 2022 @05:28PM (#31020523)
Dropbox support can rollback your Dropbox account to a previous point in time too.
Re:Inside the longest Atlassian outage by fknorangesite on Wednesday April 13, 2022 @05:31PM (#31020557)
I wouldn't expect it if I just asked. I think it's reasonable as part of their disaster recovery though.