I really disagree - physical storage in your home is the common alternative for a lot of consumer users - and that physical storage will involve maybe unplugging your external hard drive between backups - but otherwise never checking for the consistency and accuracy of the data nor the hardware. If you're working in a data center then it's your job to do these things and it doesn't take very much time... for normal folks cloud backups are likely going to be more reliable.
I've seen enough HackerNews articles:
1. My account got deleted and I can't find a human to talk to and all my pictures going back 20 years are gone!
2. A hacker guessed my password and deleted my online presence! It's all gone!
3. My storage provider went bust and all my stuff is gone!
This very story is just that - people had their own hard drive and it wasn't maintained for security so it got erased. At least this article needs to be added to your mental list of failures of local storage to balance your news-driven bias.
Two wrongs don't make a right.
If my backup harddrive is in the same house as my main storage and my house burns down, I'm fucked.
If I'm fancy, I may have multiple backend storage providers, too. Lot easier to do that then to have multiple houses. I wanted to use a safe deposit box in the past, but last time I called banks near me, they didn't even have safe deposit boxes available to rent! I can ask some friends, but then will be limited in frequency of updates, still.
(Can we make RAICSP a thing? Or do multi-cloud solution vendors already has a snappy marketing term for storing everything on all of S3, Azure Blob, and Google Storage at once?)
I strongly suspect (as in “for the right price would stake my life on it”) that there are some common failure modes across the major cloud vendors’ offerings.
Your advice is directionally sound (and is what I do for family docs and photos); I just think it has fewer meaningless 9s than you’re imagining.
I believe Resilio can do it, but it is closed source.
If you're looking for backups and not dropbox-like functionality, then something like restic might be more appropriate. It does actual content addressed backups, so you've got history for those times when you realized you messed something up six backups ago.
Just went through this with my family (in Australia) when the local bank branch said they were shutting down the service. It was basically a locked room in their branch, there was no "safe" involved.
If you care about your backups, a single cloud vendor/account vanishing isn't going to be a problem, in exactly the same way that if you care about your backups,m a single drive failure will not be a problem. They are both predictable and mitigable risks.
0.85% is extremely high. I've rolled enough 1s on d100s to know that I wouldn't take those odds!
Obviously customer support is a finite resource.
That's why companies need to start paid/premium customer support: when you lost access to your account, you should be able to pay 10, 100, 1000 of money to talk to a real person who understand how the system works and where there a mistake was made.
Like a hacker stole your account and you cannot take it back? A special person will to do a background check (e.g. call your employer) to verify that you is actually you.
Robot erroneously blocked your account because of misclassified spam-like activity? Your account is restored, and you are fully refunded.
4. My crypto exchange disappeared/went bankrupt/was a scam and now my crypto is all gone.
I swear, some people never learn.
EDIT: According to Wikipedia, ZFS, Btrfs and ReFS offer strategies to deal with various forms of data degradation, although it appears as if ZFS is still king.
I also use it for music files, in the past too many mp3s got broken.
Before I did this, in the past, I restored from backup and found I had damaged jpeg files. With par2, there's at least a recourse.
It's not stupid but redundancy is important, don't rely on cloud only.
My own anecdata: I lost a couple TBs with Streamload[0] ~2007 when they switched over to MediaMax. They wouldn't admit it was their fault and I never got any kind of compensation - even though I had just renewed my yearly subscription a couple months prior. Thankfully I had offline backups on CD/DVD but that was definitely no fun. More recently CrashPlan decided to no longer offer unlimited backup for non-enterprise users i.e. I would have to pay a LOT more money to back up multiple TBs. At least they offered me a partial refund for the remaining months in my yearly subscription. I really really hope Backblaze keeps their unlimited plan.
As for physical backups, my 8-drive RAID-5 GPT partition got somehow nuked when I was trying out a new Windows 8 installation (one of the first official builds). I never found out what actually caused it, but I immediately went back to Windows 7 and waited for 10 to be out for a couple years before finally switching over. I'm still paranoid and keep my backup server separate and off the network except to make diff backups.
Bit-rot is also becoming a serious issue. I hope to finally build a home ZFS server (with ECC RAM of course) to deal with this, as I've noticed non-streaming encoded videos sometimes just stop working after 15+ years. I'm not sure if it's actually bit-rot but I can't think of anything else that would cause this to happen.
In any case, keep multiple backups: offline and online!
Hard disk drives go on sale regularly. I replace mine usually once a year or so. They're not expensive relative to the loss of the data. I just bought a 500G SSD drive on sale for $60.
- Solid-state media...store data using electrical charges, which can slowly leak away due to imperfect insulation.
- Magnetic media...may experience data decay as bits lose their magnetic orientation. Periodic refreshing by rewriting the data can alleviate this problem. In warm/humid conditions these media, especially those poorly protected against ambient air, are prone to the physical decomposition of the storage medium.
In either case, only ZFS/Btrfs/ReFS seem to implement strategies to deal with data corruption. Kinda sad that Apple recently released a new FS and didn't bother to address this.
Cloud backup is perfectly fine as long as it's one of those copies -- I cloud backup my desktop and it saved me from a lot of loss.
But not as a backup strategy.
Nonsense. Any robust backup strategy has multiple copies, and multiple locations. Cloud is a sensible component to this.
If you use BackBlaze, they offer multiple options to restore data quicker if you're willing to shell out more money, including AWS or sending you a physical drive with your data on it, albeit the latter is kinda expensive, but if you have a ton of data and need it ASAP, is probably worth the cost.
I recently lost a 16TB drive for the second time in 6 months and my local backups were out of date by a couple months, meaning I needed to restore ~200GB of data. Aside from the web interface file explorer window being cramped (and support telling me to basically go fuck myself after suggesting that), I was able to make ZIP files from recently changed data and use their downloader tool which IMHO is a piece of crap and probably designed by some intern, but at least the download speed was fairly fast, even in Europe, in contrast to upload speeds capped at around 250 KB/s (though thankfully support multiple threads).
Data theft is another one. Encryption might help, but you have no reasonable way of knowing if the encryption is secure.
My house could burn down tomorrow, but I'd still have my cloud backups. My cloud backups could roast in a fire tomorrow, but I'd still have my local backups. And probably several other cloud backups.
If you put all your eggs in one backup, sooner or later, you're going to get screwed.
(Data transfer charges not included)
As opposed to what? I think you've got too much faith in your method.
WHAT!? How the heck did you get that conclusion from this story? These are literally your people, who backed up to a local device and essentially chose their own local NAS rather then the cloud. Which unfortunately for them came from a company with bad support that dropped it 6 years ago and likely did a shitty job even before that. But not everyone can be a tech person! Even tech people can't be that knowledgeable about more than a fraction of information technology, the field is crazy vast [0].
The failure modes here are exactly why "cloud" makes so much sense for so many. Keeping stuff up to date, or migrating if the company drops it (even noticing that support has stopped). Securing access. Verifying integrity. Amortizing the cost of different media cold storage fallbacks with things like tape robot facilities. And on and on. You wanna opine a bit on the relative probability that Amazon, Apple, Backblaze, Dropbox, Google, Microsoft, rsync.net, or a host of others just "at any moment going dark, for any reason" vs Western fucking Digital My Dropped-in-2015-Book dying? Think carefully.
I mean shit, I actually have a custom Epyc-based TrueNAS Core (not FreeNAS anymore! gotta keep up!) system with tens of terabytes as my main backing store, and replication to a remote site, with more usage of network segmentation, wireguard, ZFS user privileges so compromise of the replication credentials can't delete old stuff, then I'd expect nearly anyone to have any idea about. I still have it going to Backblaze B2 as well, and I pay to have them hang onto it for a year rather then 30 days. And the only reason I can swing it all is that I can amortize a lot of it professionally, get other usage from it, and draw upon a bunch of both knowledge and metaknowledge that my particular life path happens to have granted me.
For 99% of the people in my life "turn on iCloud/Backblaze/whatever" is the right advice. Backups are very risky without heavy automation and regular attention on top.
----
0: One thing working at various levels has given me a deep appreciation for is what broad shoulders I stand upon, many layers deep, and how valuable and deep the knowledge of so many other tradespeople I interact with in life is. The mechanics, electricians, plumbers, general contractors, structural engineers, medical folks, researchers and so on I work with or have worked with all possess skills and knowledge that I never will. That's why we're social animals. I've traded on my skills helping someone improve their network for a hand with skilled carpentry or electrical work or the like. I don't see any shame in that.
Fact: if these people had been running to a decent cloud service, they'd probably still have their data. Really "don't connect backup to the internet" is like, fractally stupid. The more one digs the more stupid angles turn up recursively similar to higher level stupid. "Don't connect backup to the internet" means what to you? What about the computer itself making the backup? If that is connected it could get infected as well, now what? Everyone is just supposed to give up on all networks entirely because "hurr durr nothing on a network is secure"? Which is wrong anyway since security an economic equation, not some ethereal absolute. I hope you never give this advice to anyone IRL.
> if these people had been running to a decent cloud service
They thought they were.
> If that is connected it could get infected as well, now what?
That's what the physical write-enable switch is for. A hardware read-only device cannot be written to.
Consider that I worked on the 757 stabilizer trim gearbox. The idea is no matter what fails, the airplane lands safely. I've been trained to think that way.
Consider also that I've worked with computers for 45 years. I've seen about every failure you can think of, including fires and floods, explosions and earthquakes, viruses and phishing.
The most unreliable, by far, part of computing is the internet.
If you are aware of the issue, and get kicked off AWS, or whatever, you move the backup. It is only a problem if your data catxhes fire and you get removed from cloud at the same time
“It’s not a backup until you restore it.”
How would you do automated backups with a physical switch? Are you proposing nobody does automated daily or more frequent backups?
Just remember that ransomware will encrypt any drive you hook up to it, including your backup drives. A physical write-enable will prevent that, and it should be disabled anytime you are restoring from backup.
I want a write-enable switch even for simple tasks like copying one drive to another, just so I don't mistakenly copy the wrong direction.
What I was alluding to but didn't make clear was that physical switches are easily defeated by human laziness or mistakes. Why have a physical switch on drives you're copying if you might accidentally switch the wrong one or eventually stop bothering? This already happens with warning popups in Windows when you try to run an untrusted program, for instance. People get trained to bypass the security because it's just a tedious obstacle.
You personally might be careful enough to forever set the switch correctly, but people who didn't even know their hard drive was years-un-patched and internet connected yet left it there to get hacked also wouldn't reliably set the switches every time either.
Regarding accidentally copying in the wrong direction, I think a safer way than a physical switch is to show some details of what will get deleted. Maybe previews of images, tiny snippets of text from the files, a graphical view of the space they occupy, etc. Make it more visceral, like throwing away an actual book where you can see how many pages it has and what the picture on the cover is.
Yep, I once reformatted the wrong drive. I plugged in a firewire drive into a friend's Mac and didn't realize he had an internal drive that apparently also showed up with a firewire icon. Whoops!
I wrote something about it [1] just a few days ago. And I have been banging on about it for years if not a whole decade.
>What we need is something like iOS TimeCapsule. Let the Phone do the Photo Management on Devices, and have an on site ( Home ) DataBackup as well as offsite ( iCloud ) Data Backup. Apple could even make iCloud Backup subscription a requirement for TimeCapsule to function.
We need both. In a simple, safe, easy to use fashion.
I don't know what you mean by "cloud", but an off-site backup is essential. Your home can be burglarized, or burnt down, or flooded. Basically anything you insurance covers for your home you should cover regarding your data.
> At any moment it could go dark, for any reason, and you have no recourse.
There is no difference between your backup going dark or your main data source going dark. The whole point is that the data is duplicated and the odds of both of them going dark at the same time is low. This only means that you should check your backup regularly.
Reminds me of the little sliders on memory cards. Though I think those were software driven. It didn’t actually lock it out from writing.
Are there any portable USB style disks that have a hardware enforced read only switch?
Home backup is a stupid idea. Any any moment your place could catch fire, and you no recourse.
I suspect your drive is more likely to die in its own that AWS to fail.
I also don't care to have the internet snooping on my data. Yes, I know about encryption. Enough to know it cannot be trusted.
I'll say again: backup drives must have a physical write enable switch on them. To all the people who argue against this - just you wait till it happens to you!
Not having multiple backups is also a very bad idea. One day my 8T drive slipped out of my hand and smacked on the floor. That was the end of it. But I wasn't out my data, just out the cost of another drive.
Cloud backup is a stupid idea. At any moment it could go dark, for any reason, and you have no recourse. Might as well bare your throat and hand some random stranger a razor.