News for hackers. Stuff that matters.
Re:Fluentd: the missing log collector by abhishektwr on Sunday November 20, 2011 @05:39AM (#3257822)
Excellent, and I am going to try it. But I still think using something like Flume has some advantages mainly because of Hadoop eco-system. For instance you can plug-in the log data to HBase and use Hive to write to high level abstracted queries and run on Hadoop. I am guessing but seems like there are plugins on the way for various systems but not Hadoop. Update: Also Flume can use any data stream, for instance Twitter stream so not limited to log Analysis only.
Re:Fluentd: the missing log collector by kzk_mover on Sunday November 20, 2011 @05:54AM (#3257844)
Yep, we agree with your opinion CURRENTLY.

We're developing Hoop (REST API for HDFS) plugin, and also Stargate (REST API for HBase). It requires additional setup at the infra-side, but still very useful and performs well.

You can feed Twitter stream by writing in_twitter plugin. That will 30 lines of Ruby code :-)