> Also, does Ceph distribute it's objects so that two drives don't contain the same set of objects? I.E. it's probabilistically impossible that a number of drives going down in a large array can wipe out all the copies of the object?

Yes, you create a CRUSH map which lets you define a hierarchical list of bucket types that reflect your failure domains (host, chassis, rack, room, datacenter, etc).

http://ceph.com/docs/master/rados/operations/crush-map/#crus...