News for hackers. Stuff that matters.
Re:Ask HN: Distributed File System by skorgu on Friday July 10, 2009 @08:01AM (#697507)
No first hand experience here but my understanding is that HDFS and Hadoop in general are geared towards high-throughput and not necessarily low latency. You'll likely have to put a caching layer in front of it if this is still [1] the case. There is already some [2] HTTP accessibility for HDFS, I'm again not 100% clear on its status but if it works well enough a nice caching proxy (varnish) with plenty of memory should help with latency on heavily-accessed files.

[1] http://www.nabble.com/Using-HDFS-to-serve-www-requests-td227...

[2] https://issues.apache.org/jira/browse/HADOOP-5010

There are no comments on this comment yet.