News for hackers. Stuff that matters.
Re:Ask HN: How to handle 50GB of transaction data each day? ... by stuartaxelowen on Wednesday February 24, 2016 @10:39PM (#11172219)
IMO investing in a hadoop system constrains you - make your data accessible via S3, Kafka, and most importantly, a good data model, and you can attach the right data processing tool when a problem that requires it comes along.
Re:Ask HN: How to handle 50GB of transaction data each day? ... by faizshah on Wednesday February 24, 2016 @10:53PM (#11172271)
I agree that a good data model is key, in fact the way you structure your data is probably the most important thing.

But the reason that the hadoop ecosystem is so good is because it's a growing ecosystem with a lot of momentum and a focus on processing large and growing amounts of data at scale and in a distributed/parallel fashion. If this is what you need the ecosystem is perfect.

Re:Ask HN: How to handle 50GB of transaction data each day? ... by haddr on Thursday February 25, 2016 @08:31AM (#11174243)
If at some point of time you use hbase, using hadoop on top of that becomes a straightforward task. Both integrate very well with each other. But I think even without hadoop you can do. If the processing is relatively light (e.g. calculating some aggregates, etc) you can do it in near real time just connecting with another consumer to kafka topic.