News for hackers. Stuff that matters.
Re:Bup: Efficient file backup system based on the git packfi... by natch on Saturday February 15, 2014 @05:57PM (#7245841)
How do people who would use this kind of thing manage to have remote servers with terabytes of available disk space on them?

Anything is possible with money, of course, but how is this anything other than really expensive?

For example AWS S3 would be $235/month (that's $2,820/year!) for 3TB not even including any data-out transfer charges. Sure there are others that are cheaper but only marginally so.

Is this really what people are doing? Makes the commercial services sound really cheap.

Re:Bup: Efficient file backup system based on the git packfi... by apenwarr on Saturday February 15, 2014 @06:00PM (#7245851)
My suggestion is to backup your cloud servers, which are expensive and redundant and have good uplink speeds, to home servers which are cheap and have good downlink speeds. You don't need your backup file server to be ultra-reliable or even up all the time, so the cheapest possible PC sitting on a home internet connection is a pretty good choice. That way, 3TB is just $150 or so plus your electricity, and it's not a per-month fee.
Re:Bup: Efficient file backup system based on the git packfi... by mverwijs on Sunday February 16, 2014 @11:16AM (#7248029)
I'm curious as to how you would restore that much data quickly, from a home-uplink to the cloud?
Re:Bup: Efficient file backup system based on the git packfi... by cbhl on Sunday February 16, 2014 @01:24PM (#7248547)
You would send the hard drive via snail-mail to Amazon.

A mass restoration is expected to be rare, so it's okay for it to be a bit more expensive.

Re:Bup: Efficient file backup system based on the git packfi... by nisa on Saturday February 15, 2014 @06:02PM (#7245859)
If you ignore the cloud services and rent dedicated servers you can get up to 6 TB disk space for 50$ a month.

edit: 45tb for 300€/month: https://www.hetzner.de/en/hosting/produkte_rootserver/xs29

Re:Bup: Efficient file backup system based on the git packfi... by drdaeman on Sunday February 16, 2014 @02:30PM (#7248839)
Unless you don't need any redundancy (cloud has some) that's more likely about 39TB (in RAID50, if controller supports such setup).

If so, that's still less than €8/TB/mo, which is better than most cloud storage providers offers. You also have some spare memory and CPU resources (so you could resell them for others as, for example, memcached instances) and a possibility to get a proper SLA, as a bonus.

Re:Bup: Efficient file backup system based on the git packfi... by _delirium on Saturday February 15, 2014 @06:12PM (#7245886)
My strategy is:

1. Regularly back up "important" directories (code/, papers/, web/, etc.) to fairly safe/redundant cloud storage with incremental history. I have pretty little of this, <50gb, so it's not super-expensive.

2. Occasionally exchange bulk but less-important backups with my brother, so we're each the other's high-latency, questionable-durability "off-site backup". No incremental dumps here, just rsync. This is where my MP3 collection, DVD rips, and similar goes.

3. Photos, which are important but also bulk, go to Flickr, which is free.

4. Don't back up stuff I can re-acquire, e.g. big public datasets I've downloaded to work on, or Debian ISOs. Also, I don't back up the OS, just my data.

There do, however, seem to be some cloud services that offer big full-disk backups for a surprisingly low flat price, e.g. http://www.backblaze.com/ is $5/mo/machine.

Re:Bup: Efficient file backup system based on the git packfi... by t0mas88 on Saturday February 15, 2014 @06:26PM (#7245948)
We use a dedicated server. It's a fairly basic machine with a single Xeon CPU and 32GB ram and a lot of drive-bays, which I think doesn't cost much. It has at the moment 16 x 3TB drives in it in RAID 50. RAID 50 is not super on performance, but fast enough to saturate gigabit in sequential operations (which backups are). So it has 32 TB of useful storage for a price of around € 700 per month (leased server at a high-end hosting company, so could be much cheaper if you buy it yourself or use a cheaper provider). Per TB that's around € 21.5 per month. Although our reasons for doing this were not based on the prices of storage in the cloud, it was based on having the data on our own machine with disk encryption and only a connection to our internal management network and not the public internet.
Re:Bup: Efficient file backup system based on the git packfi... by kbar13 on Saturday February 15, 2014 @09:42PM (#7246427)
You shouldn't think about backups as paying for $/GB. You should think of it has paying for the ability to restore correctly instantly when you want to restore. That's where the value really lies.
Re:Bup: Efficient file backup system based on the git packfi... by nathancahill on Saturday February 15, 2014 @07:14PM (#7246094)
AWS is never the cheapest option. AWS is great when you're quickly scaling up and down, but the flexibility comes at a (high) cost.

For backups, you're dealing with a relatively consistent or predictable amount of data. Buy the appropriate dedicated server for your needs.

Re:Bup: Efficient file backup system based on the git packfi... by acdha on Saturday February 15, 2014 @09:59PM (#7246472)
Remember that there's a huge difference between files you need to read and write at any time with low latency and backups which happen at larger intervals and read infrequently, with looser latency demands. For backups, the service to compare is Glacier, which is about an order of magnitude cheaper.

What I'd consider is essentially the Crashplan model: P2P / external backups locally (i.e. full LAN speed) and an off-site replica which can be cheaper and slower as long as you have a high confidence that it'll be available eventually. This way normal operations are fast but if the building burns down you're covered and presumably have higher priorities than waiting for a restore to run.

Re:Bup: Efficient file backup system based on the git packfi... by sitkack on Saturday February 15, 2014 @06:21PM (#7245921)
AWS is for startups with venture capital, ephemeral storage and compute, or someone needing to deploy a high traffic website instantly.