Re:Ask HN: Distributed File System
by simonw on Friday July 10, 2009 @06:45AM
(#697433)
If you want a GFS clone, you /really/ want HDFS from the Hadoop project - I'm reading the new O'Reilly book on Hadoop on the moment which is excellent.
[ Parent
| on HN ]
No first hand experience here but my understanding is that HDFS and Hadoop in general are geared towards high-throughput and not necessarily low latency. You'll likely have to put a caching layer in front of it if this is still [1] the case. There is already some [2] HTTP accessibility for HDFS, I'm again not 100% clear on its status but if it works well enough a nice caching proxy (varnish) with plenty of memory should help with latency on heavily-accessed files.
[1] http://www.nabble.com/Using-HDFS-to-serve-www-requests-td227...