News for hackers. Stuff that matters.
Re:Wuala: Secure Cloud Storage by a3_nm on Monday June 10, 2013 @10:58AM (#5854829)
> is it worth the trouble if Wuala's clunky client takes 100 times longer than Dropbox to sync a file between two devices

Is this some general FUD about encryption being slow, or an actual measurement?

Maybe all client-side encrypted solutions happen to be crappily implemented, but I don't think that client-side encryption would necessarily lead to something noticeably slower.

Re:Wuala: Secure Cloud Storage by kijin on Monday June 10, 2013 @12:17PM (#5855364)
I have absolutely no intention to spread FUD about client-side encryption, and in fact I don't think Dropbox's speed advantage has anything to do with (lack of) encryption. Encryption shouldn't have any noticeable impact on sync speed, since modern CPUs can encrypt a file a hundred times over in the time it takes to upload/download it. I agree with you and suspect that it has more to do with crappy architecture.

Anecdote: SpiderOak takes 5 minutes synchronizing metadata with the server if I start it on a laptop that I only use occasionally. That's how long it takes for the client to find out what files I changed in my other machines in the meantime, including all the files that have nothing to do with this particular machine. Nothing gets synched during this time, and the client sometimes consumes a whole CPU core for extended periods. It's very annoying when I just want to see a few files that I know I changed in another machine. I understand that the protocol needs to be different if all the encryption is done by the client, but this is just ridiculous. Wuala is no better in my experience, although YMMV depending on your use case and filesystem structure. (I have over 50GB of data in my SpiderOak account, made up of a quarter million files. At one point I had over 100GB stored in there, and it was a PITA to sync any files at all.)

As an interesting comparison, Microsoft SkyDrive performs just as terribly as SpiderOak does when you give it a few hundred folders with ~100K files to sync, despite the fact that it does no client-side encryption. It just sits idle for half an hour or more, scarcely utilizing any network resources. I have no idea what it's doing. Maybe it does sleep(random()); between each upload. In any case, encryption isn't making it slow. It's just bad app design, bad protocol design, and perhaps overloaded servers.

Dropbox, on the other hand, immediately begins to upload/download my files, and once it's done scanning any large folders, its speed is only limited by the bandwidth of my Internet connection. When I need to do a presentation, I can just work on the .pptx file in my Dropbox on my desktop PC, take my laptop to the conference room, and expect the .pptx to be there as soon as I wake my laptop. That's the meaning of convenience. By the same metric, none of Dropbox's commercial competitors (perhaps with the exception of Google Drive, which I don't use) are even close to convenient.

Re:Wuala: Secure Cloud Storage by rarrrrrr on Monday June 10, 2013 @02:00PM (#5855935)
SpiderOak cofounder here.

It's not really about the encryption directly; it's about the way the database design has to change to support zero knowledge (the server can no longer do the database work.)  Things like garbage collection have to happen client side.  Described in detail here:  https://spideroak.com/blog/20091026143000-why-and-how-spider...

FYI, when SpiderOak first starts running, like any backup software, it has to scan your filesystem to see what may have changed while it was shutdown, and that can take a lot of IO and CPU if you have many folders and/or many small files (hundreds of thousands).  (SpiderOak works with any folders you like, including externa/network drives, not just one folder like Dropbox.)  

In a profile, basically none of this CPU time is spent in the crypto module, just mostly for unserializing the per-folder journal to compare state.  SpiderOak does some optimizations with directory hashing to avoid having to open the journal to scan for specific changes in unchanged folders, but there are some situations where that optimization is defeated (such as if you remount a file system with changed gids, inode IDs, timestamps, etc.)  

Anyway, once it's running, it uses the operating system facilities on Linux/Mac/Windows to notice changes automatically so rescanning the whole backup selection after startup is not usually necessary.

In any case, thanks for your interest in SpiderOak. :)

Re:Wuala: Secure Cloud Storage by Pyramids on Tuesday June 11, 2013 @12:32AM (#5859750)
Hello Alan,

I have a quick question for you which might be of interest to others as well:

Why isn't SpiderOak open source yet?

I've read your FAQ answer[1] on this, however it doesn't really give a concrete explanation, besides "soon" and "licencing concerns" which is definitely disappointing.

I currently use my own, EncFS based sync solution, however I'd love to be able to use SpiderOak, or a third party open source application which supported syncing with SpiderOak (if you'd ever consider exposing an API/Protocol which allowed such.)

This is the only reason I'm not using your service currently, however I love the concept and hope you'll take it into consideration. It's likely I'm not the only one with similar concerns.

[1]: https://spideroak.com/faq/questions/35/why_isnt_spideroak_op...

Re:Wuala: Secure Cloud Storage by kijin on Tuesday June 11, 2013 @01:14AM (#5859878)
My hard drives are reasonably fast, so even with a quarter million files it doesn't take long to scan the filesystem. I have other backup software that scan the exact same folders and finish in less than 30 seconds. So I suspect that it's your journaling/unserializing system that takes up the bulk of the startup time.

Fortunately, I only need to start up SpiderOak once in a while, because nowadays I put my computers into hibernation instead of shutting them down and starting them up again. I also noticed that the more often I use SpiderOak, the less time it takes to start up, presumably because there's less delta to process. Also, as you said, once SpiderOak is up and running, it's relatively fast. Rest assured that the performance issues, although annoying, have not dissuaded me from renewing my SpiderOak Plus (100GB) account once again last month.

I also like the fact that SpiderOak's "Queue" and "Log" screens tell me exactly what it's doing at any given moment. Waiting becomes a lot more tolerable when I know what I'm waiting for. I hate backup tools that assume the user is too dumb to understand what's going on behind the scenes. Even Wuala's Upload/Download queue never seems to work properly.

Re:Wuala: Secure Cloud Storage by objectivefs on Monday June 10, 2013 @01:48PM (#5855877)
Client-side encryption can be very fast when done right. For your use case a filesystem stored in the cloud might be a better solution, that way only the data you are using is downloaded to each machine and the writes are done immediately. Give ObjectiveFS[1] a try, the free preview is available for a few more days, and see how fast client-side encryption can be.

[1] https://objectivefs.com

Re:Wuala: Secure Cloud Storage by kijin on Tuesday June 11, 2013 @01:17AM (#5859890)
Interesting, but an all-cloud solution would seem to be a better fit for servers with fat pipes (especially EC2) than for individuals with crappy residential connections. I have fiber at home, though, so I might try it out sometime.
Re:Wuala: Secure Cloud Storage by fakeer on Monday June 10, 2013 @11:57PM (#5859588)
I wouldn't go into technicalities because I can't - mostly because it's not my area of expertise and I've never looked into it deeply or maybe not even from the surface other than a user's perspective. The end user.

Wuala is seriously slow and clunky piece of software. Period.