News for hackers. Stuff that matters.
Re:Ask HN: Distributed File System by simonw on Friday July 10, 2009 @06:45AM (#697433)
If you want a GFS clone, you /really/ want HDFS from the Hadoop project - I'm reading the new O'Reilly book on Hadoop on the moment which is excellent.
Re:Ask HN: Distributed File System by skorgu on Friday July 10, 2009 @08:01AM (#697507)
No first hand experience here but my understanding is that HDFS and Hadoop in general are geared towards high-throughput and not necessarily low latency. You'll likely have to put a caching layer in front of it if this is still [1] the case. There is already some [2] HTTP accessibility for HDFS, I'm again not 100% clear on its status but if it works well enough a nice caching proxy (varnish) with plenty of memory should help with latency on heavily-accessed files.

[1] http://www.nabble.com/Using-HDFS-to-serve-www-requests-td227...

[2] https://issues.apache.org/jira/browse/HADOOP-5010

Re:Ask HN: Distributed File System by Tichy on Friday July 10, 2009 @07:13AM (#697454)
That is probably what I meant, too - I read that one of those OS projects had a clone of the GFS, only better (supposedly). Must have been Hadoop, as I haven't looked at many others.