News for hackers. Stuff that matters.
Re:The AeroFS Private Cloud by HorizonXP on Tuesday December 10, 2013 @12:52PM (#6882268)
So I'm actually setting up a private cloud for my company right now. It's a company of 1, so the requirements are quite extensive. :-P

I'm building a FreeNAS server with 6x3TB hard drives in a RAID-Z2 config. My goal is to allow my Mac to use it for Time Machine backups, but to also use AeroFS as my file sync mechanism when I'm both in the office and on the road.

Hopefully it works out smoothly. I'll have to figure out how to access the machine from behind my router, and I'll have to determine how to get it to automatically back up to S3 + Glacier. I think there's going to be a lot of details that I'll have to research here.

Re:The AeroFS Private Cloud by pedrocr on Tuesday December 10, 2013 @01:42PM (#6882683)
>My goal is to allow my Mac to use it for Time Machine backups

Note that this is completely unreliable. Time Machine isn't built to work over the network. The network support is badly hacked together and will destroy your backups on network faults. They basically treat the network volume as a block device that gets corrupted when the connection fails. Using it over wifi makes this painfully obvious.

[deleted]
Re:The AeroFS Private Cloud by pedrocr on Tuesday December 10, 2013 @02:11PM (#6882909)
I'm positive about Time Machine over AFP. iSCSI was indeed pointed out as a potential avenue with less issues, but if I recall didn't really fix it completely. If you're running wired+iSCSI you're probably much less prone to this. Try and yank some network cables during backup and see how it handles it.

I ran into more issues than most because the particular AP I have at home doesn't work well with the macbook air I was trying to backup and resulted in a lot of disconnects. So it was a bit of a stress test but the issue is real and well documented. You can find instructions all over the web on how to fsck sparsebundles to try and get back your backups[1].

This is particularly annoying because Time Machine is supposed to be a user friendly version of versioned backups like you can do with rsync+hardlinks. I have a script to do that for me on my personal linux machine and would love to be able to deploy TimeMachine+AFP for the rest of the family. Unfortunately apple decided it really needed to have hard-linked directories for just this case and so decided that the way to do that over the network was to have a single binary blob with a HFS image on the server instead of the proper files in the filesystem (filesystems don't usually support hard linked directories, HFS was hacked to do it). That way you not only lose direct access to the individual files on the server you also lose the whole backups when there's disk corruption...

I should probably have a look again to see if iSCSI is a viable solution to this.

[1] http://www.garth.org/archives/2011,08,27,169,fix-time-machin...

Re:The AeroFS Private Cloud by kika on Tuesday December 10, 2013 @03:08PM (#6883317)
I can confirm that. I've lost a TM backup recently made over the (wired) network. The cable failed, backup in progress hung, TM volume remained locked until I reboot, I rebooted and sparsebundle got screwed up. TM started it all over again.
Re:The AeroFS Private Cloud by pedrocr on Tuesday December 10, 2013 @03:13PM (#6883356)
Check the link in my post. If you still have the sparsebundle you can probably recover it with an fsck.
Re:The AeroFS Private Cloud by kika on Tuesday December 10, 2013 @03:18PM (#6883395)
Well, luckily I didn't _need_ this backup, so I just deleted the bundle and started over.
Re:The AeroFS Private Cloud by nostromo on Tuesday December 10, 2013 @01:57PM (#6882792)
Why not just use BTSync for syncing? I use it for this purpose and it's fantastic.

For backup you can use one of many solutions to archive to S3. I wouldn't recommend Time Machine.

Re:The AeroFS Private Cloud by mindslight on Tuesday December 10, 2013 @02:02PM (#6882825)
Why not use Unison? It only does pairwise synchronization, but is Free software and extremely mature.

(sigh, it seems the tech community has become blind to anything that isn't heavily blogvertised.)

Re:The AeroFS Private Cloud by porker on Tuesday December 10, 2013 @03:33PM (#6883486)
> (sigh, it seems the tech community has become blind to anything that isn't heavily blogvertised.)

There's so much released and so much noise, it's hard to keep track of all the good things that come out - especially if they're not mentioned within their tribe of friends/followed ppl.

I don't know how to get round that - the web is huge and no one can keep track (or have a network that touches) all of the open source releases.

Re:The AeroFS Private Cloud by yapcguy on Tuesday December 10, 2013 @09:00PM (#6885402)
Unison is no longer under active development, right? Would love to use it but going forward, is there a big enough community to fix bugs, add new features etc?

http://www.cis.upenn.edu/~bcpierce/unison/status.html

Re:The AeroFS Private Cloud by mindslight on Wednesday December 11, 2013 @12:37PM (#6889085)
It has not been a problem in my experience. There isn't major development (proper support for inotify and flexible topology would be nice), but Unison does what it says extremely solidly. There's certainly an active user community and the occasional bugfix/feature. Reread that notice again concentrating on the good parts, and take a look at the changelog. There's certainly activity, it's just not time intensive - this is both the blessing and apparent curse of using a high level language.
Re:The AeroFS Private Cloud by chrisrogers on Tuesday December 10, 2013 @02:30PM (#6883052)
Unison requires a bit more administrative work to set up and maintain than BTSync, in my experience. People really like the Dropbox experience.
Re:The AeroFS Private Cloud by karanbhangui on Tuesday December 10, 2013 @04:45PM (#6883969)
Had high hopes for BTSync, and we used it for several months on recommendation of a colleague. However, we found it was fairly buggy in terms of handling files with non-alpha numeric characters in it, and files just wouldn't sync. We suspect it might have been a rogue path parser for .SyncIgnore, not following unix file conventions (we were using it on OSX and Linux), but recently moved over to AeroFS and haven't had a problem since.
Re:The AeroFS Private Cloud by HorizonXP on Tuesday December 10, 2013 @01:59PM (#6882803)
I do like BTSync and considered it, but I'm not the biggest fan of the idea that my data is stored on everyone's computer. AeroFS allows me some semblance of control, and as I grow, it will allow me to add users easily.

I won't know until I try it though, I could just as easily change my mind and use BTSync.

Re:The AeroFS Private Cloud by ketralnis on Tuesday December 10, 2013 @03:45PM (#6883564)
It doesn't put your data on everyone's computer. The data goes onto the computers that are sharing the folder. If you have two computers, the data is on two computers.
Re:The AeroFS Private Cloud by kika on Tuesday December 10, 2013 @03:11PM (#6883347)
BTSync could be installed everywhere, but it only syncs between the folders with the same key. So if only your computers have your key(s) then only they would have contents of your folder(s).
Re:The AeroFS Private Cloud by toomuchtodo on Tuesday December 10, 2013 @01:03PM (#6882345)
Re:The AeroFS Private Cloud by bifrost on Tuesday December 10, 2013 @01:18PM (#6882477)
Unfortunately it leaves out encrypting the backup. I think there is a tarsnap plugin for FreeNAS which does encrypt the data...
Re:The AeroFS Private Cloud by barrkel on Tuesday December 10, 2013 @04:07PM (#6883699)
Check out the recovery probabilities of raidz2 vs raid10. You're winning ~17% more space at the cost of a substantial performance hit, and almost no difference in reliability, depending on how you estimate HDD failure rates. Raidz2 is not fast when everything is working well, and if any of your drives develop a speed problem, the whole array will be reduced to that speed.

I have a 11x4T raidz2, and will be migrating to 3x4x4T raid10. My last array was 2x4x1.5T raidz1, 3 drive failures in its 5 year lifetime, but not concurrent so no data lost.

I use CrashPlan to get data from one place to another; I back up NAS to CrashPlan, PCs to CrashPlan, and PCs to NAS. If you don't use CrashPlan's app to back up to their cloud, it doesn't stop you from using it to back up to other machines you own. And it works on Windows, Mac, Linux, Solaris, and with a bit of hacking, FreeBSD.

Re:The AeroFS Private Cloud by phlo on Wednesday December 11, 2013 @03:31AM (#6886589)
Could you elaborate a bit on your CrashPlan setup? I have a somewhat similar setup and have been trying to back up my NAS (Synology DS1812+) to CrashPlan for about half a year now. I was able to start the backup, but the transfer rate quickly dropped to about 1.2 Mbps or lower, and having backed up about 1.5 TB the CrashPlan client now won't even run a backup.

Are you using Crashplan+ or any of the Pro/Enterprise plans? What sort of transfer rates do you get? And were you able to complete a full backup of that 11x4T array?

Re:The AeroFS Private Cloud by barrkel on Wednesday December 11, 2013 @05:00AM (#6886809)
I'm using CrashPlan+ Family Unlimited (it's a home NAS!); I also bought a separate CrashPlan+ license when I first signed up. I don't have more than 1TB backed up specifically to CrashPlan as yet though. A big chunk of that NAS is for future growth, and a lot of it is a playground for experimenting with serving up media that we either have physical copies of or is of low replacement value.

I do see CrashPlan updating at a leisurely rate, but that's OK with me, as my upload bandwidth is limited. I'd be interested to see if it completely chokes past a certain point, like you say it does for you. But it's not yet at the point where I need to worry about it.

Re:The AeroFS Private Cloud by bestham on Tuesday December 10, 2013 @07:28PM (#6885002)
While you might be right about the small space advantage, I would feel more safe if the position of the failing drive (and the next failing drive) does not matter at all.

If a drive develops a speed problem, I'll replace it.

Re:The AeroFS Private Cloud by barrkel on Wednesday December 11, 2013 @01:03AM (#6886163)
You might feel more safe, but you're not as safe as you feel.

Suppose that once one drive has failed, the chance of another drive failing (at least an URE, but potentially worse) while reading it entirely (as necessary during a scrub / resilver) is 10%. I don't think this is very implausible, as RAID HDD failures are often correlated, and not only because it's inconvenient to distribute HDD purchases.

The chance of successfully recovering in a mirrored scenario is 90%. You only need to read one drive.

In raidz1 with 6 drives, your probability is 0.9^5 - you need to get lucky 5 times, because you need to read every other drive fully to recover. That brings you down to less than 60%. Raidz1 with 5 drives is 0.9^4 - a bit over 65%.

Raidz2 for 6 drives combines these two stats. The probability of one failing disk while reading is 1-0.9^5. But you can recover from that. And the probability of that recovery is 0.9^4. So your consecutive probability of failures is (1-0.9^5)(1-0.9^4). Subtract that from 1 and you get your probability of getting lucky.

Work out the numbers, and you'll find it's only 86%. It's less than mirroring.

There are a couple of simplifications in the maths above (don't forget, a second drive failure means another drive to resilver, and a restart to the procedure), and a big assumption based on reliability %. But frankly, unless you think HDD failure is already fairly rare - and it's not - mirroring is usually ahead on safety, and the lead only increases with number of disks in the array.

I used to think like you do, that I'd feel more safe if a single disk failed, knowing I could still tolerate another failure. That's why my array is currently raidz2. But I worked out the numbers, and it changed my mind. And that's why my next array is going to be raid10.

Not to mention that it's hard to get much more than 150M/sec for any realistic I/O with raidz2. I have over 1G/sec of potential HDD bandwidth. Unless I'm doing a scrub, I can't touch the potential performance. And of course all disks act like they're on the same spindle, there is zero random I/O advantage for raidz.