Is this some general FUD about encryption being slow, or an actual measurement?
Maybe all client-side encrypted solutions happen to be crappily implemented, but I don't think that client-side encryption would necessarily lead to something noticeably slower.
Anecdote: SpiderOak takes 5 minutes synchronizing metadata with the server if I start it on a laptop that I only use occasionally. That's how long it takes for the client to find out what files I changed in my other machines in the meantime, including all the files that have nothing to do with this particular machine. Nothing gets synched during this time, and the client sometimes consumes a whole CPU core for extended periods. It's very annoying when I just want to see a few files that I know I changed in another machine. I understand that the protocol needs to be different if all the encryption is done by the client, but this is just ridiculous. Wuala is no better in my experience, although YMMV depending on your use case and filesystem structure. (I have over 50GB of data in my SpiderOak account, made up of a quarter million files. At one point I had over 100GB stored in there, and it was a PITA to sync any files at all.)
As an interesting comparison, Microsoft SkyDrive performs just as terribly as SpiderOak does when you give it a few hundred folders with ~100K files to sync, despite the fact that it does no client-side encryption. It just sits idle for half an hour or more, scarcely utilizing any network resources. I have no idea what it's doing. Maybe it does sleep(random()); between each upload. In any case, encryption isn't making it slow. It's just bad app design, bad protocol design, and perhaps overloaded servers.
Dropbox, on the other hand, immediately begins to upload/download my files, and once it's done scanning any large folders, its speed is only limited by the bandwidth of my Internet connection. When I need to do a presentation, I can just work on the .pptx file in my Dropbox on my desktop PC, take my laptop to the conference room, and expect the .pptx to be there as soon as I wake my laptop. That's the meaning of convenience. By the same metric, none of Dropbox's commercial competitors (perhaps with the exception of Google Drive, which I don't use) are even close to convenient.
It's not really about the encryption directly; it's about the way the database design has to change to support zero knowledge (the server can no longer do the database work.) Things like garbage collection have to happen client side. Described in detail here: https://spideroak.com/blog/20091026143000-why-and-how-spider...
FYI, when SpiderOak first starts running, like any backup software, it has to scan your filesystem to see what may have changed while it was shutdown, and that can take a lot of IO and CPU if you have many folders and/or many small files (hundreds of thousands). (SpiderOak works with any folders you like, including externa/network drives, not just one folder like Dropbox.)
In a profile, basically none of this CPU time is spent in the crypto module, just mostly for unserializing the per-folder journal to compare state. SpiderOak does some optimizations with directory hashing to avoid having to open the journal to scan for specific changes in unchanged folders, but there are some situations where that optimization is defeated (such as if you remount a file system with changed gids, inode IDs, timestamps, etc.)
Anyway, once it's running, it uses the operating system facilities on Linux/Mac/Windows to notice changes automatically so rescanning the whole backup selection after startup is not usually necessary.
In any case, thanks for your interest in SpiderOak. :)
I have a quick question for you which might be of interest to others as well:
Why isn't SpiderOak open source yet?
I've read your FAQ answer[1] on this, however it doesn't really give a concrete explanation, besides "soon" and "licencing concerns" which is definitely disappointing.
I currently use my own, EncFS based sync solution, however I'd love to be able to use SpiderOak, or a third party open source application which supported syncing with SpiderOak (if you'd ever consider exposing an API/Protocol which allowed such.)
This is the only reason I'm not using your service currently, however I love the concept and hope you'll take it into consideration. It's likely I'm not the only one with similar concerns.
[1]: https://spideroak.com/faq/questions/35/why_isnt_spideroak_op...
Fortunately, I only need to start up SpiderOak once in a while, because nowadays I put my computers into hibernation instead of shutting them down and starting them up again. I also noticed that the more often I use SpiderOak, the less time it takes to start up, presumably because there's less delta to process. Also, as you said, once SpiderOak is up and running, it's relatively fast. Rest assured that the performance issues, although annoying, have not dissuaded me from renewing my SpiderOak Plus (100GB) account once again last month.
I also like the fact that SpiderOak's "Queue" and "Log" screens tell me exactly what it's doing at any given moment. Waiting becomes a lot more tolerable when I know what I'm waiting for. I hate backup tools that assume the user is too dumb to understand what's going on behind the scenes. Even Wuala's Upload/Download queue never seems to work properly.
Wuala is seriously slow and clunky piece of software. Period.
I have no idea what the performance difference is (and clearly , neither do you), but the answer to your question is "yes". For some people, at least. It's a must-have feature for some people, not a nice-to-have-as-long-as-it-doesn't-inconvenience-me-too-much feature. Some people have data that they'd like to keep synced across systems that they simply can't keep in unencrypted form on a third-party server, no matter how much trust they have in the company that owns the service.
Of course, there are a bunch of different ways to keep your data secure and still have it be available to you on different systems, and non-open-sourced cloud storage with client-side encryption is still an imperfect solution, but it is a solution that is perfectly reasonable for many people.
The only drawback I have encountered is that Dropbox limits the path names to 255 characters. With encrypted file/folder names this can become an issue especially with some Java projects.
I think this really gives me the best of both worlds. Security for the files I think need it and the Dropbox infrastructure for syncing files.
When it comes to performance, a layer of encryption on top of Dropbox is much better than any competing solution that has client-side encryption built into it, because you're still using Dropbox's awesome infrastructure and lightening-fast sync protocol.
But I ran into the same problem as you did. The 255-char limit sucks, and it's completely unnecessary because even Windows/NTFS allows you to use longer paths (up to 32KB) if you know which APIs to use. I suppose fixing this is not a priority for Dropbox because you normally don't run into it unless you use encryption, and Dropbox has no reason to encourage the use of encryption among their users.
http://en.wikipedia.org/wiki/Truecrypt#Reasonable_paranoia
"""It is challenging to create binaries from source code that match the official binaries bit-by-bit for purposes of verifying their integrity due to compiler options
...
There has been no known comprehensive review of the source code by a qualified cryptographer.[46][44] Thorough security code review and testing is hard, tedious, and painstaking work, and very few people have the skills to do it. There was, however, a functional evaluation of the deniability of hidden volumes in an earlier version of TrueCrypt by Schneier et al. that found security leaks.[47]"""
The underlying problem is that you're trusting closed client software from a third-party. Once that is running on your system, arguably it's game over as far as what Dropbox or its like can and can't do to you.
(If you trust them, ok. But when documents state that "they're next", many of us start to wonder. And again, this isn't just access to a server. Through their client software, this is access to your machine, that you've already granted.
Admittedly, I'm describing the typical Dropbox user scenario and not where the client is manually applying -- or not -- each update, and somehow supposedly gaining insight into the capabilities of each update/version before doing so. And I'm also ignoring various forms of restricting the access of the Dropbox client, although if they really want you, they and parties using them would have extensive resources to apply to breaking out of such constraints.)
FYI this article suggests to use encfs for an encrypted folder in your dropbox, this comment https://news.ycombinator.com/item?id=5855465 suggests that this might not be a wise choice, so you might want to (re)consider that part.
Let me take the opportunity to also thank you for writing up your email best practices. I hope to implement them or something similar for myself.
One of the challenges for me in all this is keeping up including with the implementation details. It feels akin to swimming upstream, with the current outpacing me.
No, the crypto in TrueCrypt doesn't amplify the number of changed blocks. But it does disperse the changed blocks throughout the encrypted image, and this does result in higher bandwidth used during the sync (at least with DropBox). Here's an analysis I did a few years ago:
http://www.lacie.com/us/company/news/news.htm?id=10722
Now, client-side encryption is a much more interesting aspect of their service, but is it worth the trouble if Wuala's clunky client takes 100 times longer than Dropbox to sync a file between two devices? Ditto for SpiderOak, JungleDisk, and every other backup/sync solution that I've used so far that boasts client-side encryption. And it wasn't due to Dropbox's deduplication, either. Some of them just talked lazily with the server for several minutes before they even started to upload/download any files.