Ceph is a very interesting project. RADOS, their distributed block store is now mainline I believe and the project is coming along in leaps and bounds.

I'm unsure if anybody has a large scale RADOS based blob store though. It would be interesting to see how it holds up.

Dreamhost was doing a ceph beta with testers over a year ago. I see there's a number of blog posts about ceph on their site.