I'm also backing up my photo archive on B2 via rclone and it works great, it costs like $1 for 200 GB which is awesome. But I'm not encrypting the files.

Encryption raises the barrier for both third-parties and family. In case something happens to me, I want the technical barrier to be low enough for my family to discover the backup. Another reason is that in my experience encrypted data is more sensitive to bit rot and bugs than unencrypted data. I'm backing up important work stuff with Arq Backup for example and I've had my archive corrupted once. Not sure if it was the software's fault or the storage.

My rule of thumb is ... if the data should be discovered by my family in case I'm not around, then I won't encrypt it. Photos are not worth encrypting anyway, since a lot of them end up being shared on Facebook, Flickr, Instagram, etc, as photos are meant to be shared, at least with your family.

That said I still expect Backblaze or Dropbox to keep my data private. Not secret, but private and there is a difference.

I've thought about that a fair a lot recently myself, and came to a different conclusion, but like yours as a counterpoint. What I've been considering was going analog and writing down how to get to the few places I've kept my secrets and employing something like a Yubikey to access an admin account on my computer and paper instructions in an envelope in the family safety deposit box. It's not a complex as a two-person thing I read about once, but seems like a nice compromise where I can still feel like I'm protected, but the nuclear keys exist if someone ever needed them.

Where do you draw the line around data that needs to be discovered? I'm thinking about instructions to access things like bank accounts or such that they may or may not already have access to, where I'd want them encrypted but accessible. Not that I've got secret Cayman accounts or anything, but financials are usually things i want heavily encrypted, but do want family access to in case of the worst.

I normally think of financials as things that don't need to be heavily encrypted, because we have laws limiting liability in case financial information is stolen and misused. What makes you feel it needs encryption?

Not in all cases and even if thier are legal remedies it takes months to clear it up and while you are clearing up the effects of the fraud, the person who has stolen your identity is committing new fraud until they get caught.

I know people that were arrested for fraud because their identity was stolen and someone else was committing financial crimes in thier name. They always have to carry official police reports with them just in case they get arrested again.

Honestly, I don't know, I've just always held banking info as something that must be treated as such. That said, it's not like I don't use credit cards on the Internet, but something about someone getting my bank account numbers does worry me. Plenty of people who know me well know my mother's maiden name, for example, and those two pieces of info together could spell trouble. I'm also in the camp of unless it needs to be shared, may as well treat it as private, and that includes encryption.

If it were true then an identity theft would be called a bank robbery https://youtu.be/CS9ptA3Ya9E

That's an interesting viewpoint to strike a balance between privacy and secrecy.

Can you shed some light on how you share the photos with non-technical family and friends, given that B2 has no app as such?

I have some experience with AWS/Azure and both of them do not support folders, and the workaround is to have slashes in the filename to create a virtual directory. Is it the same with B2?

I'm only using B2 for backup and it's doing a fine job, since from what I understand it also does unlimited history of the files, so in case files get corrupted, theoretically I still have previous versions around.

I keep my photos on Dropbox too, which is how I share them with family, besides sending files over WhatsApp, which is popular these days. But they only provide the history of changes for 1 month, or 3 months for Pro. As has been said before, solutions like Dropbox are not reliable for doing backups without specialized software like Rclone or Arq Backup, that can keep a version history.

My archive is currently less than 150 GB, so B2 is really cheap. I also have an offline backup on a portable hard drive. The idea with backups is that if you have data you care about, then it's a good idea to have at least 2 backups in different locations, made via different software.

> I have some experience with AWS/Azure and both of them do not support folders, and the workaround is to have slashes in the filename to create a virtual directory. Is it the same with B2?

B2 has folders, you can navigate them in the online interface. That said the service doesn't have polished apps available, being a platform like S3. It has no desktop or mobile apps currently. Although if they survive, given its price, I'm sure apps will happen at some point.

B2 doesn't have folders.

The online interface simply assumes that a slash in the filename should be represented as a folder; and they encourage apps to do the same. I believe they also enforce a max distance between slashes that is smaller than the max filename length.

What this means is that their is no way to, for instance, query what the root directories are, short of listing all files.

If you have a directory, you can list its contents using a prefix search (although the prefix need not be a directory, and this will not just list the toplevel elements)

Disclaimer: I work at Backblaze.

>their is no way to, for instance, query what the root directories are, short of listing all files

This is not true! Try this from the b2.py command line:

b2.py ls <bucketName>

That would list all the top level folders. The APIs are designed to support two things: 1) listing all files, or 2) navigating and listing the contents of each folder.

> it also does unlimited history of the files

Yes, but you're paying for that storage. If you sync 100GB of photos then locally make a small EXIF data change to all of them and sync again, you're now paying for 200GB of storage. B2 has Lifecycle Rules [0] to help keep versions from getting out of control and the API has methods for handling versions for clients like rclone [1] to use.

B2 doesn't have it's own desktop app but 3rd party desktop apps like Cyberduck use the API work with B2.

[0] https://www.backblaze.com/blog/backblaze-b2-lifecycle-rules/ [1] https://rclone.org/b2/#versions

I would like to point out that given [1] in the US that means online services don't agree with your balance. Anything that is hosted is accessible to law enforcement, and potentially to anyone that has a legal disagreement with you.

[1] https://en.wikipedia.org/wiki/Discovery_(law)

In the spirit of sharing details of open source tools that work well with B2, I am using restic and finding it to work well.

Apparently restic now supports using rclone as a backend:

https://restic.net/blog/2018-04-01/rclone-backend

+1 for restic. I switched to restic(+B2) from duply+duplicity(+S3) (another backup tool supporting dumb remote storage, encryption, incremental backups, and snapshots) and life is so much better. Duplicity needs the duply front-end for too many basics, it regularly needs full backups to be made and stored, it's not built at all for random access (it takes eons in order to list or fetch a specific file from a specific incremental backup), and it needs a barely documented command line switch in order to not bug out with S3 if you have too many files (why is that option not default? Its default S3 configuration has a max limit on file size. Duplicity splits the archives containing your files, but it has one un-split archive listing filenames or some kind of metadata that it doesn't support splitting, so if that file gets too big, then your backups just start failing). Restic is nice.

rclone is excellent piece of code and has solved a large part of issue for my own project. I maintain a slightly modified fork of rclone [1] and integrated that in daptin [2] as a server side piece. So I can seamlessly work with any cloud storage for assets/uploads thru daptin.

I wrote up a walk-thru some time back. The changes basically include replacing all the "fatal logs" with "error logs". I keep merging the upstream back regularly.

[1] https://github.com/artpar/rclone

[2] https://github.com/daptin/daptin

[3] https://medium.com/@012parth/daptin-walk-through-oauth2-goog...

I'll definitely give this a try.

I'm currently backing up about 1.7TB of pictures to B2 from my Qnap NAS. Qnap has a backup app called Hybrid Backup Sync.

The problem is, while doing the one-way upload sync, the Qnap app downloaded a lot of data as well. I got confused why I was showing a lot of 'b2_download_file_by_name' API calls on the Backblaze reports page (600 GB upload resulted in 700 GB download calls).

I've also opened a thread on the forum - https://forum.qnap.com/viewtopic.php?p=673557

Contacted Qnap support and they said a little bit of download is normal but this looks abnormal. Logs are all fine on the Qnap so they suggested I contact Backblaze.

I wonder if anyone else has faced this.

Why not use $60 you're paying per year for 1Tb of online storage to buy a 2Tb hard drive and use it for the backup?

It's a "cheap, easy, reliable: pick any two" situation.

Cheap and easy: buy a 2 TB drive and keep it at home. If some disaster affects your home -- flood, fire, burglary -- it can take out your data and its backup.

Cheap and reliable: buy a 2 TB hard drive and keep it somewhere else. Keeping the backup up-to-date means regularly bringing the drove home, updating it, and putting it back.

Easy and reliable: pay for a service like Backblaze that automatically backs up all your files to a remote server.

There are other benefits to services like B2 especially, namely being able to access your backed-up files from any device or location, or being able to link people to your files on a high-speed server.

Or...

You put the 2 TB drive somewhere else (at a relative's) and keep it updated regularly via network.

That's my set up (but with a bigger drive).

At home, I have the master copy of the data on my file server. Then I have backup #1 that is in the same location and backup #2 that is in a different location.

Both #1 and #2 get updated at night with a "timemachine-like" backup system based on rsnapshot. The network traffic goes over ssh.

Remote backup system #2 cost a UPS, a RaspberryPi and an 8 TB drive, which is about ~$250-$300 total.

The initial sync is best done locally of course, but deltas can generally easily go over network at night.

Cheap, reliable, and (relatively) easy (if you're a geek, that is).

I remember there being software back in the day that did exactly this.

The name escapes me right now, but basically you had to add "friends" in the software, then dedicate a certain amount of HDD space to it. It would then back up your files to your friends' computers, and theirs to yours. Backups were encrypted so your friends wouldn't be able to see your files.

It was a super neat idea, I wish I could remember what it was called so I could see if they're still around...

There's an open source one called Tahoe: https://tahoe-lafs.org/trac/tahoe-lafs There used to be a company with a more usable similar product called allmydata, but it seems to be defunct.

What happens if you move/rename a bunch of big files?

Nothing bad, if you use "bup" or "borg"; The latter has better delete support, so is a better choice for rolling backup if you sometimes delete data. "bup" has the advantage that its repo format is git, which makes it easier to hack.

Both use rsync-style deltas to only send changes, but they use a content-addressable scheme like git so renames are a small metadata change record.

Also, both offer ftp and fuse interfaces if you need to access an older backup.

Bad things! rsync isn't smart enough (AFAIK) to know that files have been renamed or moved: it just sees files disappearing on one side and appearing on the other side, so the daily delta can get big.

Yeah that sucks…

I'm looking into using the incremental diff/snapshot feature of btrfs to implement a more efficient solution :P

DRBD may be a good solution to this problem, although I haven't spent the time to see what it would take to replicate over ssh, and the kind of traffic that is incurred vs changes in origin.

https://en.wikipedia.org/wiki/Distributed_Replicated_Block_D...

Interesting! Never heard of this before.

Disclaimer: I work at Backblaze.

> Why not use $60 you're paying per year for 1Tb of online storage to buy a 2Tb hard drive and use it for the backup?

We HIGHLY recommend both. There is a philosophy called the 3-2-1 philosophy of backups. You should always have three copies of the data, two onsite, and one remote. https://www.backblaze.com/blog/the-3-2-1-backup-strategy/

Online/offsite backup is a different use case. They are paying $60/year so that, if their house burns down, gets flooded, disk gets fried by lightning, they still have their family pictures.

Local backup is cheap and fast, and you should do it too. But it doesn't provide geographic redundancy.

Another $60 buys a water and fireproof safe for the hard drive. I assume it also helps against lightning :-) I honestly think remote backup is an overkill for personal needs and new risks you get by placing your valuable data on someone else's hard drive are not always internalized.

And you take your non redundantly stored hard drive out in 5 years and you can’t retrieve data on it.

But realistically, isn’t it worth $60 a year to have a constantly backed up hard drive? The alternative is to take the hard drive out of the safe every so often and do a backup and put it back.

I've recently recovered data from a pair of 15GB and 20GB drives I last used in 2001 (and were stored in an ordinary closet of a house that experienced inside temperatures ranging from 5C to 30C over that time, and great humidity/dryness flactuations over those 17 years). There were 16 bad 512-byte sectors on one of the drives, but otherwise all worked.

Modern higher density drives are probably less resilient, and who knows how flash drives will fair after 17 years in the closet - but my experience so far is that HDDs trump backup tapes on every measure including costs except at extreme sizes (at this point in time, into the petabytes).

I've hard a few hard drives fail in the past decades but I've always been able to retrieve the data. I'm far less confident in a remote company.

Your point is valid and constant, seamless backup is indeed a good thing to have. Whether it's worth $60/year is one's own decision.

> I've hard a few hard drives fail in the past decades but I've always been able to retrieve the data

Care to share what kind of procedures you use?

I've recovered some data for friends and employers who want it back but aren't prepared to pay > USD1000 for it but if I cannot connect to the disk I'm lost.

(My tricks: tilting the disk, freezing the disk, leaving ssds powered on but sata unconnected, and even before that photorec and ddrescue etc.)

Note: don't do any of the above if data needs to be recovered at any cost, in that case just contact a data recovery company.

By putting your drive in a safe, you now have to unlock it, take it out, plug it in, sync, unplug it, and lock it away. Which is a hassle if you want to stick to regular (hourly? daily?) backups.

It is proven that if you introduce friction into process, over time that process will be followed less.

The other point to remember is BB is not for archiving. BB states this itself. At least with external drives it is possible to archive.

True, the standard Backblaze offering will erase your backup if you’re not online for six months. If you have an external drive that’s offline for 30 days, they will erase your backup.

A B2 based backup solution costs more but you don’t have those limitations.

Accord to Backblaze the retention period is thirty days for the personal plan [1]. Where do you see six months? If a file is deleted on the source device, and the deletion is synchronized to the BB repository, the recovery window is 30 days.

[1] https://help.backblaze.com/hc/en-us/articles/217666628-What-...

Different thing: https://help.backblaze.com/hc/en-us/articles/217664898-What-... If your computer is offline for six months and doesn’t connect to BackBlaze, they erase all of your data.

If you have an externally attached drive on your computer and it isn’t connected in 30 days and your computer is online, they erase that backup of the external drive.

If you reconnect your computer after a month and you don’t have the external drive connected, they erase your backup.

That was my reasoning as well until burglars took my NAS (in addition to the laptops, of course.)

It's much more easy and reliable this way.

I perform hourly backups of my VPS and personal computers, storing it all into a giant repo on OVH and B2. If my house goes up in flames, I have to redo, at worst, 1 hour of work.

Additionally I won't have to deal with expanding to a 4TB drive eventually.

I use Backblaze's standard service. I look at it as cheap belt and suspenders insurance. I do onsite backups using Time Machine--along with the occasional sync to another drive.It's offsite and it's a completely independent backup mechanism. $50/year or so is essentially not worth worrying about in this context.

WRT your other comment: Yes, there's some small level of incremental security risk but there's so little that's genuinely sensitive in my storage, I'm willing to take that risk. And, yes, it's probably overkill but for the cost, there are a lot of things I spend money on that are probably unnecessary :-)

Disclaimer: I work at Backblaze.

> I do onsite backups using Time Machine, and also Backblaze.

You are doing everything correctly. You are following the 3-2-1 backup philosophy, which is: "3 copies of the data, 2 copies locally, 1 copy remote". Here is a blog post we wrote about it: https://www.backblaze.com/blog/the-3-2-1-backup-strategy/

Hard drives fail and you need them physically to access the data. Online backups allow you access everywhere and are generally much more reliable than a single, local disk.

That was my previous solution. It worked well for a couple years until the disk crashed and I lost some of my data :-(

What do you do when someone spills coffee on your drive?

Presumably so that the backup is offsite.

rclone is vulnerable to data exfiltration attacks

https://www.danieldent.com/blog/restless-vulnerability-non-b...

Am I reading this right? Google/B2/... might send your data to another URL you didn't expect.

Not sure why that matters, or why it's an attack. Since they have your data anyway, as that's the whole point of the service, to store your data on their hard drives. Why go through the trouble of sending it elsewhere? To play games with your data for giggles?

No, the API can tell your software to send some private LAN files, e.g. some IP-filtered secret NFS store, to an URL of it's choosing (so to itself, or your competitor). This is bad, as long as you don't heavily jail and firewall the software to prevent it from ever accessing anything it shouldn't (need to).

I quickly skimmed, but this entire attack is assuming that the attacker has successfully MITMed the API. At that point everything is already nuked, so of course you can fabricate any number of attacks. Did I miss something important?

Cool setup. I'm not a Backblaze customer, so I'm curious. How is your setup better than using their client for personal backup? The webpage mentions threading and encryption.

Backblaze's backup product doesn't support backing up a NAS. It is for backing up a single Windows or Mac computer, and priced as such. They state this policy is to avoid abuse. Fair enough.

Backblaze's object storage product, B2, is priced per GB-month, so you pay for what you use. Fair enough. Because it is charged this way, it is open for whatever creative use developers can come up with.

I use B2 because I'm locked out of using Backblaze Online Backup - and that's fine with me, because it's the right product for the job.

That's interesting. I have been considering a similar setup. Are you running this from Linux?

Yes, I do all this on Linux, but they support other environments too (most of the environments Go can build for): https://rclone.org/downloads/