News for hackers. Stuff that matters.
Re:GFS: Evolution on Fast-forward by vicaya on Wednesday August 12, 2009 @06:57PM (#759481)
I think I independently came up with the same idea to solve the distributed gfs master problem: use a separate bigtable/gfs1 cluster for master metadata for gfs2.

I'm glad that I'm not crazy :)

Re:GFS: Evolution on Fast-forward by seymourz on Thursday August 13, 2009 @05:49AM (#760392)
Also, does the gfs1 still have single master, so that the gfs2 has mulitple Bigtable tablets serving as distributed masters for gfs2? Is this the cause for "In fact, it just makes the bottleneck limitations of the system’s single-master design more apparent than would otherwise be the case.", as stated in the article?
Re:GFS: Evolution on Fast-forward by vicaya on Saturday August 15, 2009 @12:53PM (#764896)
gfs1 is still single master, but the workload is much simpler in this case: it serves the gfs2 master bigtable cluster exclusively. Most of the documented gfs master failures are due to misbehaved map-reduce clients. Also the gfs1 master can be down for extended period of time without affecting the master operations, due to the nature of the cluster (you're unlikely to create a million files per second resulting in much compaction and splits in metadata tablets)

The quote you mentioned actually meant that if you use Bigtable on top of gfs1, the single master failure is more apparent due to the low latency requirement of the application that use the Bigtable.

Re:GFS: Evolution on Fast-forward by seymourz on Tuesday August 18, 2009 @03:42AM (#769978)
Is this vacaya related to the vacaya of hypertable? :-)
Re:GFS: Evolution on Fast-forward by seymourz on Thursday August 13, 2009 @04:21AM (#760285)
Ahaa , we had just conceived this way several months ago in China ...