I'm sorry we weren't able to determine the cause of the slowness for you. Troubleshooting an end-to-end encrypted product is hard because you can't just see everything that's happening by looking at the server. We have seen ISPs deny or aggressively throttle connections to our destination networks. Palo Alto firewalls classify traffic to SpiderOak as an online backup service and often block it outright or put it at least priority. I'm not saying that these were necessarily the causes in your situation.
SpiderOak keeps improving and the 2017 road map is action packed. If for some reason you would ever like to try SpiderOak again on me, you're welcome to contact me directly, or write to [email protected] where these days we do a pretty good job of taking care of everyone. Otherwise I'm glad you've found backup solutions you're happy with. Cheers!
> I think perhaps you might mean 2015 instead of 2016.
Looking back, yeah, you're right. Sorry about that.
> If I've found the correct case, we did at least suspend billing when the issue started and eventually issued a full refund.
Yep, that sounds like me.
> If for some reason you would ever like to try SpiderOak again on me, you're welcome to contact me directly, or write to [email protected] where these days we do a pretty good job of taking care of everyone.
I've downloaded the latest client and I'm running it on OS X right now. There doesn't appear to be any change. The client just sits for long stretches of time doing absolutely nothing, just like it used to (no disk activity, no CPU, no network activity). The UI shows a bunch of pending "actions" which the logs show the server has already confirmed. I'll gather some more details and email you the results tomorrow at some point.
EDIT: I've tested with a dedicated server with a gigabit Hurricane Electric pipe (notably, HE peers with WANSecurity, the AS that announces SpiderOak's IPs) and I'm able to get 80Mb/s. On my 30Mbit Comcast connection at home, I'm able to get around half speed (though I'm able to saturate it to other services). I've emailed you more details but this is more for the benefit of others.
While I'd prefer the ability to saturate my pipe completely, this is good enough that it's no longer a huge problem for me.
As a customer / user, it's always good to have additional feedback channels.
> It would be really great if more CEO's and personnel could come here to have honest discussions and respond to information about things they're particularly knowledgeable about.
If it's at the expense of having an actual support channel then that's be terrible. More and more the only way to get a response from a companies is to make a stink on some form of social media (including HN).
I'm not saying SpiderOak is in that category (haven't used them so can't say). But responding to Q&A about your product on HN is not a substitute for responding to email, chat, phone, or having a feedback forum on your own site.
> a co-founder at SpiderOak: A "zero knowledge" encrypted, space efficient, multi-computer, perpetual offsite backup, sync, and sharing solution
https://spideroak.com/manual/spideroak-on-mobile
quickly found using:
https://www.google.com/search?q=site%3Aspideroak.com+zero+kn...
I don't know how I would react, but from a comment I think I'd like being told soon, even if it was a failure (in that case it's not even clear if you were failing or if it was some system in the middle) from a company I paid. I'd be less angry after 3 days with a disappointing news than after a month.
Then it's just me; I don't know how other feel about this. I'd be curious to know though.
Thats my main issues with spideroak.
I'm glad your customer satisfaction scores are higher, but I'd rather not share any of my information with you. I'm not the op, but if I was, I would feel rather violated.
Well then don't choose a public forum to vent a complaint, and since you're not the OP it doesn't matter anyway.
Turnabout is fair play: if you go into a thread about a product and make a strong play that their customer service sucks and then an officer of that company steps in and does what they can to see if there is a way to re-engage you on their dime that's about as good as you could possibly expect. On top of that he did not volunteer any info that wasn't already in the OP's post except for a possible correction of the date.
So you're wrong, twice.
FWIW absolutely no affiliation whatsoever with Spideroak.
For what it's worth, we do have very strict policy about what information regular customer service staff can access or share publicly (i.e. none in most cases) and how someone who calls in must establish beyond a reasonable doubt that they really are the original customer before we will communicate with them. We're very careful about allowing any customer data to exist in 3rd party systems. We don't even use Google Analytics. https://medium.com/@mccamon/yeah-we-ditched-google-2fa644578...
That said, many people also expect customer support delivered over Twitter so some flexibility is required. In this case I made a judgement call and decided to answer.
All-in-all this hardly seems like a reason to feel "violated"...
If he'd named my ISP, location, address or something like that I'd say it's inappropriate but I think this is totally find and I actually appreciate that he went to the trouble.
If you have a realistic option (e.g. a fully supported solution) from another provider I'm all ears, but frankly SpiderOak has been good in that they have lost 0 data for me and have been able to successfully restore several times. If they could just double or triple the speed for uploading it'd be a huge boon to me, but realistically most of the places I've gone in the world being able to upload 2MB/s is faster than my local Internet is capable of.
I think Borg would be a great solution for my use at home where I have a NAS on my LAN to provide local storage for the repository. But while traveling the world with my MacBook it doesn't cut it.
Arq looks good from that angle but I think would quickly cause my costs to overrun me. Having good backups is what gives me the confidence as a photographer to overcome my inner pack rat. Without good backups I'd never be able to ruthlessly delete images from my collection that don't exceed "okay" into "great". But because I know my images are backed up effectively forever I can really be honest in my art.
I personally use Arq against Google Cloud Storage. I have around 1TB, and I use it to back up several external drives in addition to my MacBook. (Arq is nice in that it backs up the drive if it's connected, but won't prune olds backups if it's not, unlike services such as Backblaze.)
I think I pay around $6-7/mo. I use a coldline bucket, which is the cheapest kind of bucket. Restoring is super fast (unlike Amazon Glacier, which I used previously) and not too expensive.
I wouldn't risk my amazon account on it, personally.
There are plenty of people who are storing absurd amounts of data [1]. If Amazon were cracking down on this, you'd hear it in /r/DataHoarder first.
The ToS [2] don't mention any size limitations. In fact, it says you can store pretty much any file as long as the data doesn't violate any laws, including copyright.
The only really worrying part is this:
3.2 Usage Restrictions and Limits. The Services are offered
in the United States. We may restrict access from
other locations. There may be limits on the types of
content you can store and share using the Services,
such as file types we do not support, and on the
number or type of devices you can use to access the
Services. We may impose other restrictions on use
of the Services.
[2] https://www.amazon.com/gp/help/customer/display.html/?nodeId...
- Client side encryption, with my key (and trust that Arq doesn't phone home). If that is an issue just pre encrypt the data before Arq backs it up.
- Use a storage provider of my choice, i.e. I'm confident that it is secure with Amazon S3 - no unknown shortcuttings.
- Multiple destinations possible.
- Open source tool capable of decrypting the data.
- Scanning of new / changed files takes could be faster and less demanding for the file system (on my Macs, Finder sometimes temporarily freezes when Arq is scanning for new / changed files)
- Mail reports for '0 errors' (all the time, so the mail reports for errors only are rather useless)
- Running in user context only, i.e., if you log out while your Mac is still running, the backup will not continue
- Loading of existing backups (reading / caching index) tends to be slow (and sometimes hangs)
- GUI does not scale up for many existing backups
With that being sad, you listed the major reasons to use Arq and all in all, I am a happy Arq user.
Switched to Sync.com a Toronto based company, with only Canadian servers. That means a lot to me as I've tried to remove myself from storing on American servers (just out of good practice on my part), and the service and support has been spectacular. I tried other options as well and this won me over.
Unreliable and slow completely describes my experience with both Backblaze (lost the data for one of my drives which unfortunately I learned after the drive failed) and Crashplan (the Java client brings my rMBP to a crawl when it runs). There has got to be a decent online backup service that's reliable, reasonably priced, and respectful of resources out there somewhere.
Two that have been recommended to me recently that I haven't tried yet but would like to are Arq Backup [1] and Tarsnap [2].
(Used Backblaze for one month happily during trial period, decided not to switch because of their, imho ridiculous, file/version retention/deletion policy. And an unhappy Crashplan customer for ~5 years actively looking around for similar feature set but sanely usable client and and to switch to. Arq doesn't fit the bill. I wish Backblaze wouldn't delete the files/versions/disconnected drives that soon if not not ever. And also website only restore actually is funny)
(I of course understand that Backblaze wants to make its service as easy as possible, to they have other defaults.)
`borg prune -v --list $REPOSITORY --prefix '{hostname}-' --keep-daily=7 --keep-weekly=4 --keep-monthly=6`
Now, if I just had laptops to backup I would have chose deja-dup over borg because I could have configured it in seconds.
A nice project would be to code a TUI front-end to borg init/create/prune/check commands that is as quick to setup as deja-dup and automatically adds systemd units or cron jobs to the system, with an alert triggered when something fails.
Yes I tried backupninja with borg but encountered some problems with it.
Totally a great option if you do have live storage to backup to but personally I don't have a full server worth of disk spare at any given time.
When I purchased my SpiderOak account everything seemed to be going well. The service is twice as expensive as BackBlaze and CrashPlan but has better security. The software UI is also quite intuitive. The only problem was the support. A couple of weeks into my subscription my client died. I’d try to restart it and it would die again. I looked into the problem and then I emailed support at SpiderOak. Then I waited. Then I got an email from someone at SpiderOak saying that an ad campaign really paid off and they were signing up a lot of clients. They’d get back to me.
They never did.
I sent other emails. Sent my log files in. I was very, very polite. Over a week went by. Finally, I asked told them I’d really rather not work with them any more.
More time went by.
Finally, someone got back to me. They gave me instructions on how to cancel my account. They said they were sorry but they understood. Then they also told me how to fix my original problem. I would have stayed with SpiderOak if they had done something, anything more than that. I realize margins are tight but surely they could have thought a little bit about how to give me some confidence in their support. Instead, I was left with the distinct impression that their support staff is either chronically understaffed, under qualified or just doesn’t give a damn. Either way, it made me realize that this was not a company I could count on if something really went wrong.
Either way, send an email today - we'll be watching.
It seems to me these providers are basically a web hoster that specializes in storage, which means 24/7 support, very high uptime, capital investment in infrastructure and ongoing maintenance and planned upgrades, and technical specialization for writing and supporting custom software. Many web hosters get away with low prices by not guaranteeing data integrity and putting a premium on large amounts of storage (or simply over-selling capacity). It's very hard for me to imagine how a company can have a large number of customers, each with 1TB of highly redundant data with high service uptime and immediate support, for $12/month.
For just photos, I would buy the second or third to cheapest option and be done with it, but it still gives me the willies that these prices are so low. The prices of different providers seems almost spastic, running from $1.50 to $150 for a terabyte of backup. Unless there's very good reasons for the variance in price, something seems fishy.
The standard market rate is actually closer to $10/TB and that's from the big providers like Google.
> It's very hard for me to imagine how a company can have a large number of customers, each with 1TB of highly redundant data with high service uptime and immediate support, for $12/month.
It's actually not as hard as you think. I built and colocate my own 56TB rackmount and over 4 or so years (the warranty on the drives) it works out to around $2.5 per raw TB, inclusive of all hardware and bandwidth. Optimize that for storage (a lot of my server costs are compute), scale it up, assume the server will live 8 years or so until you replace it and you should be able to get that below a dollar per raw TB. Replicate a few times and you're done.
> The prices of different providers seems almost spastic, running from $1.50 to $150 for a terabyte of backup. Unless there's very good reasons for the variance in price, something seems fishy.
The reasons are variations in replication (for example 3 copies vs 2 copies vs no copies vs 10+3 erasure coding), support, location and bandwidth prices I imagine.
Some of them (I remember Tarsnap's page about this) outsource the storage itself to AWS.
https://aws.amazon.com/s3/pricing/
$12 per TB per month would give a slight profit margin on the S3 Standard - Infrequent Access class (currently, a profit of about $2.20 per TB per month) and a significant profit margin on Glacier (about $10.90), with an appreciable loss (about $6.50) if the average customer block exceeded the Infrequent Access restrictions. It might be possible to make the client optimize accesses in some way that makes it less likely that many blocks will exceed Infrequent Access rules, and maybe even to try to keep the typical block in Glacier instead?
https://aws.amazon.com/s3/faqs/#sia_anchor
It seems like an interesting challenge. (Yes, support and administrative costs need to come out of that profit margin.)
Also, Amazon apparently offers "lifecycle policies" to try to do this automatically instead of explicitly. It seems like those policies try to migrate blocks into a cheaper tier that assumes less frequent access if the blocks have not, in fact, been accessed on a certain schedule. A cloud storage vendor using AWS as its backend could then try to optimize manually at the per-user level, or just let AWS do it itself empirically, which wouldn't save quite as much money as correct guesses about what particular users will do with particular data, but would require minimal engineering effort on the vendor's part.
Edit: It seems like it will be hard to compete with the AWS/S3 backend for this kind of service! Now I wonder if there are providers who are known not to use S3.
These days, I use a mix of Backblaze and Arq (with Amazon Cloud Drive).
Backblaze is great but versioning is limited to 30 days. More versioning is probably not feasible, i.e., other cloud backup providers with unlimited versioning have to find other ways to keep their data usage under control.
[1] https://news.ycombinator.com/item?id=13306745 [2] https://www.duplicati.com/
I agree its slow compared to other providers, but its seems reliable to me, both what it uploads and testing restores.
Right now I have a small backup selection (around 90 gigs) but most of the time I had been running with a approx 800 gigabyte backup set on my mac and except for slow initial upload (I am located in europe so that might also factor in) everything have been running fine.
I backup to a 1 TB storage VPS from time4vps.eu, which at 72 EUR for two years is also much cheaper than Crashplan which I used before (and which seemed to be far more resource hungry on my Mac).
Only downside of Duplicati in my opinion is that it performs daily backups not "real-time" backups as some others do.
On the other hand, the post above asks for a "full service solution", in that case I would probably still recommend CrashPlan - it also supports local encryption with your own encryption keys and has been very reliable for me as well.
For example the hubic contract [1] states that bandwidth is limited to 10 Mbit/s upstream and downstream. Compare this to 400 Mbps dedicated port speed for each Storage server at time4vps.eu. Indeed, time4vps.eu offers 32 TB of bandwidth per month with the 4 TB storage plan, while hubic's 10 Mbit/s means that even if I transfer 24/7 for the whole month I'll only be able to transfer 3.3 TB of the 10 TB they offer.
For heavy personal use something like Amazon Drive [2] is probably a better choice, because it offers unlimited storage for $60/year. People on reddit are saying they have hundreds of TBs stored with no complaints.
--
I'll post my usual story:
In February 2016, SpiderOak dropped its pricing to $12/month for 1TB of data. Having several hundred gigabytes of photos to backup I took advantage and bought a year long subscription ($129). I had access to a symmetric gigabit fibre connection so I connected, set up the SpiderOak client and started uploading.
However I noticed something odd. According to my Mac's activity monitor, SpiderOak was only uploading in short bursts [0] of ~2MB/s. I did some test uploads to other services (Google Drive, Amazon) to verify that things were fine with my connection (they were) and then contacted support (Feb 10).
What followed was nearly 6 months of "support", first claiming that it might be a server side issue and moving me "to a new host" (Feb 17) then when that didn't resolve my issue, they ignored me for a couple of months then handed me over to an engineer (Apr 28) who told me: "we may have your uploads running at the maximum speed we can offer you at the moment. Additional changes to storage network configuration will not improve the situation much. There is an overhead limitation when the client encrypts, deduplicates, and compresses the files you are uploading"
At this point I ran a basic test (cat /dev/urandom | gzip -c | openssl enc -aes-256-cbc -pass pass:spideroak | pv | shasum -a 256 > /dev/zero) that showed my laptop was easily capable of hashing and encrypting the data much faster than SpiderOak was handling it (Apr 30) after which I was simply ignored for a full month until I opened another ticket asking for a refund (Jul 9).
I really love the idea of secure, private storage but SpiderOak's client is barely functional and their customer support is rather bad.
If you want a service like theirs, I'd suggest rolling your own. Rclone [1] and Syncany [2] are both open source and support end to end encryption and a variety of storage backends.
[0]: http://i.imgur.com/XEvhIop.png
[1]: http://rclone.org/
[2]: https://www.syncany.org/