I want to like Backblaze I love their harddrive report but this is not helping.
Since the software version no longer works, they wouldn't be able to do a restore anyway. Keeping their old data on the live servers longer could be a liability.
(just a guess, but i imagine it's something along these lines)
The Backblaze software doesn't handle restores, it just backs things up. To restore, you download zip files of selected folders from the website.
I agree that a better solution would have been to remove the backup to a secondary system that the client could later access if needed.
Deleting in 7 days is unreasonable.
It wouldn't have been unreasonable to move away from SHA1 in that time period, for example. They may have needed the original files to get a SHA256 hash.
I think it must have been blocking some organizational objective. I can't think of any reason why they would bother to send that email, much less to go all the way to the CTO to send that email.
1) You are paying for a backup service.
2) We decided that the version of your software is too old, even though it backed-up data.
3) Since you are a valued customer, we will delete your data.
4) We are a HAPPY company! Have a great day!For example if you plug in an external harddrive and back it up with Backblaze, if you don't plug it in at least once every 30 days, then its cloud copy is deleted. So no, the intention of Backblaze is not to keep your offline machine from several years ago securely backed up.
That being said, you can pay for unlimited revisions which just stores your data in b2 and you pay per month on the extra storage. In that case then yes I would expect the data from years ago to still be present. But that's a recent addition, and not what the OP was using.
source: https://help.backblaze.com/hc/en-us/articles/217664898-What-...
Suppose my house catches on fire. I might not immediately get around to booting all of my computers again (in fact, it might be months until I have the same set of computers again), but I sure don't expect Backblaze to just start deleting files even while I'm paying them!
Importantly they had the _choice_ to delete this data. Note that the email seems to indicate that the sender will personally delete the backup in a week.
Yes, they could have proactively canceled my bill. Maybe they would have! We'll never know, because I responded to this email right away.
He responded to my questions in a lengthy email, which I'll paste again here because I had the foresight to ask in advance for future permission to post publicly.
===
Hi Robbie, thanks for getting back to me! It's hard getting through the email marketing overload. :-)
> Would you be amenable to holding off on deleting the backup > for two weeks (instead of one)
No worries, the most important part is that you are aware the computer is no longer actually performing fresh backups of new data. We can wait a while.
> would you mind if I published your email publicly?
Go ahead, and one of our marketing guys (Yev) already saw your tweet go by, it made everybody in our office laugh too. I'm off in my corner watching logs and emailing the random old customer, they didn't know I do this. :-)
One of the things I've been amazed at doing backup software is how many assumptions I made at the start that aren't true and the MODERN Backblaze software works around:
1) assumption: your disk on your laptop will reliably store bytes or fail. Wrong, it can throw random bits, undetected unless Backblaze keeps its own parallel check sums.
2) assumption: laptop RAM works reliably. Wrong, I think there is a reason servers run ECC RAM, and I'm confused why many laptops choose not to. I guess saving $5 is more important than data Integrity?
3) assumption: the HTTP posts either work or fail. Wrong: WAY MORE OFTEN than the file system are undetected silent data corruption. Solution: client end to end SHA hashes.
4) assumption: customers are sane and won't "edit" the Backblaze internal data structures sitting on the customer's laptop. Nope! If customers are computer beginners they tend to behave nicely, and the most advanced customers are very careful, but there is this crazy middle ground who know enough to be dangerous to themselves. :-)
I really really think that the most modern version of Backblaze (2.3.1 and later) is quite stable and might run flawlessly for a decade on a computer, but some of those early versions like anything starting with a 1.0 or 1.5 scare me because there are too many assumptions in them, not paranoid enough. Yes, the old versions WILL WORK for 99 percent of customers, but at our customer numbers that means every day we have to explain why a restore failed to some poor guy who chose to edit his Backblaze data structures corrupting his own backup and we didn't detect it soon enough.
Our most recent client versions are deeply, profoundly paranoid software that basically assumes your laptop is right smack dab in the middle of melting down, throwing disk and RAM errors as an insane laptop owner not only ignores the warning signs of impending data Armageddon, but fights with Backblaze trying to defeat it from backing up because the customer is worried Backblaze is "wasting his bandwidth". I feel good about this, it's like my whole career was training up to write this incredibly important piece of software.
-- Brian Wilson. CTO Backblaze [email protected]
I am touched you saved my email and remembered me, and that at the time you didn't respond negatively to the message. Thank you for being a customer.
The only thing I regret is that comment about version number of 2.3.1 and the claim that it would run flawlessly for a decade. Oh geez, please upgrade to 8.0.1, you won't regret it.
-- Brian Wilson, CTO Backblaze, [email protected]
e.g. https://www.reddit.com/r/backblaze/comments/jsrqoz/personal_...
It seems that they may be changing their tune, at least as of a few months ago: https://www.reddit.com/r/backblaze/comments/m3xynu/backblaze...
The current limit is 50 characters. There are reasons we don't let it get super long like 2,000 characters but we could easily double or triple the maximum length it if it was requested. Is there a number of characters you have in mind? I found a chart here: https://i.imgur.com/trR2u8g.jpg that says a randomly generated 17 character password would take 93 trillion years to crack. The chart doesn't even go up to 50 characters. Backblaze also supports 2-factor authentication via SMS or Google Authenticator codes which personally I would highly recommend to customers also.
> who knows what kind of encryption you guys are using
Backblaze has two product lines: "Backblaze Backup" where we wrote the app that encrypts data on your laptop before uploading into our storage cloud, and "Backblaze B2" where you use any third party tool (or write your own) so the encryption in that case is all under your control. You can see a list of 3rd party tools here: https://www.backblaze.com/b2/integrations.html (make sure you scroll down). There are little pictures of penguins by the software that supports Linux, little pictures of an apple for software that support Macintosh, and a little Window icon for the software that support Windows.
For "Backblaze Backup", we use a well known design where we use symmetric AES-128 to encrypt the file, but each file is encrypted with a different AES-128 key, and that key and an Initialization Vector (IV) is then encrypted by a 2048 bit RSA public/private key. You can read about the encryption flow we use in this blog post: https://www.backblaze.com/blog/how-to-make-strong-encryption...
>The current limit is 50 characters. There are reasons we don't let it get super long like 2,000 characters but we could easily double or triple the maximum length it if it was requested. Is there a number of characters you have in mind? I found a chart here: https://i.imgur.com/trR2u8g.jpg that says a randomly generated 17 character password would take 93 trillion years to crack.
Usually the reason people want longer "password" options is because they're using a diceware style pass phrase of 4-6 randomly generated words. It's not so much about ultimate security as many people having trouble memorizing decent fully random passwords. Key stretching and multifactor helps further of course. Longest normal English words are something like 21 characters before getting into niche science terms, so very conservatively 160 characters is probably going to fit even an extremely conservative edge case passphrase. That'd be 8x of some of the very longest words (unlikely to be randomly chosen) and match a 128-bit key which is plenty [0].
>* Backblaze also supports 2-factor authentication via SMS or Google Authenticator codes which personally I would highly recommend to customers also.*
I hope you'll consider adding (via Webauthn online or OS or other toolkits to the application) standalone hardware token support (like Yubikeys or Nitrokeys). Those are enormously more secure, convenient, and also have the bonus of eliminating another set of 3rd party dependencies. If you looked at those in the past and skipped them due to lack of device support worth looking again, every major platform should have coverage at this point.
----
0: Also, rather then thinking in terms of "years to crack" which inherently depends on the processor power thrown at it, might be better off considering in terms of combinations (or entropy). A standard symmetric key at this point is 256 bits, which means 2^256 combinations, but with stretching 128 bits is probably a very very safe seed for the password component. So that could be a "password" of [01] 128 characters long. That'd be roughly equivalent to [0-9] 38 characters long, or alphanumeric cased (26*2+10=62) 22 characters long. Current Oxford English dictionary edition I think has around 170000 words. If we figure a diceware program throws out ultra short, ultra long, and ultra weird words for convenience (whole point is something people can remember) so maybe it's cut down to the 70000 most common words >3 characters, that'd take 8. At a more typical 4-6 words, equivalent to maybe a 17 character cased alphanumeric password.
Gotcha. I'll file a Jira ticket to see if we can easily and quickly raise it to 200 characters (round number). I think we can handle that easily and safely.
> standalone hardware token support (like Yubikeys or Nitrokeys).
I'll run the idea by a few groups and see what people say. For feature requests NOT related to security, traditionally it's kind of a customer voting system. If enough customers want a certain feature it sorts to the top of the priority list and gets done sooner.
For security things it is obviously different, like if a big exploit like Heartbleed is released in the world we drop everything else and focus on that.
This is sort of somewhere in the middle, it's security related so it gets some additional boost up the priority list for that, but it's also a security feature not all customers would choose to use, so it is still affected by whether our sales and support teams are hearing requests for it.
Thanks so much for the reply, and yeah that's done-forever territory. Even "just" tripling to 150 as you earlier suggested would almost certainly be plenty, into pathologic edge cases.
>I'll run the idea by a few groups and see what people say. For feature requests NOT related to security, traditionally it's kind of a customer voting system. If enough customers want a certain feature it sorts to the top of the priority list and gets done sooner.
>For security things it is obviously different, like if a big exploit like Heartbleed is released in the world we drop everything else and focus on that.
Sensible way to do it. And yeah I agree this is kind of a hybrid of both. It's not a flaw per se, but it's more than a feature enhancement as there are genuine security implications. If you haven't looked into it, there have been cases of SMS hijacking for example, you can find plenty of discussions on HN alone with a quick "sms hijack" search. It's better than nothing but SMS infra just is fundamentally not very secure. One time tokens with a seed are better, but less convenient or secure then a blackbox that can enforce operator presence requirements or at least completely sequester keys.
Even a couple of years ago it would have been more annoying to integrate but the standards are progressing well enough now that it's probably worth another look. Adoption of hardware keys in business also finally seems to be hitting critical mass, though if we're in an S-curve at last this time around it's well before the steep slope.
Anyway, thanks again and best of luck with everything! As it happens plan to link up a new TrueNAS installation with B2 starting tonight :).
You can't have it both ways, demanding privacy and respect from software publishers then crying when the software doesn't serve your use case exactly and make assumptions about your intentions.
I wonder whether it’s their marketing and PR spin/strategy. Apple’s I’d definitely think so.
And no I don’t hate the companies itself. I am a paying b2 customer and I am satisfied with that service and I’m an iPhone mini user I (kinda) like the phone at that size and price; but moving away from these will be just one or a half medium to major fuckup away.
I have rarely read any fandom about Backblaze. There may be just a few mentions whenever there is a topic of backup or S3 alternative.
But I remember vividly there were at least a few hundred if not thousands of hateful comments on HN when they swear to god they will never use them again because of a mistake they made with Facebook analytics. Anyone who defended Backblaze at the time for its possible engineering mistakes were borderline being attacked. I dont know if there is any Stockholm syndrome on Backblaze but I definitely see Facebookphobia on HN and on Twitter.
You’ve never met people who use iPhone or a Mac? What planet are you from?
Backblaze + Dropbox have saved me multiple times.