Don't use Hadoop when your data isn't that big (2013)

Re:Don't use Hadoop when your data isn't that big (2013) by phd514 on Tuesday May 23, 2017 @12:35PM (#14402685)

I've worked in a similar situation -- a transaction processing system with ~1M tx/day and ~2TB total data. They used 15 Cassandra nodes and a monster Redshift cluster (>64 CPU cores, ~512GB RAM, 30+TB disk) for the OLTP and OLAP queries. I almost can't put Cassandra and Redshift on my resume because when more knowledgeable people ask about my experience with them, that pathetic use case makes me look bad by association.

[ Parent | on HN ]

Re:Don't use Hadoop when your data isn't that big (2013) by scapecast on Tuesday May 23, 2017 @01:23PM (#14403114)

good lord, why did they need a cluster with 30TB when all they had was 2TB? you could downsize that cluster by a factor of 10x.

[ Parent | on HN ]