Re:Building a Streaming Analytics Data Stack
by teej on Tuesday November 10, 2015 @12:59PM
(#10541010)
Let's get this out of the way - I love it when companies are open and transparent about their architecture. Sharing intimate details like this is fantastic.
Where I'm struggling is that there are a number of questionable choices here with little justification. For example, why a HTTP front-end? This is fine for webhooks but I'm not going to let my website's backend open an HTTP connection for every event I want to send out. The decision to store the data in Elasticsearch and Cassandra is equally dubious. In my experience Elasticsearch has been a maintenance nightmare and has not been a perform any and robust reporting solution at scale.
Where I'm struggling is that there are a number of questionable choices here with little justification. For example, why a HTTP front-end? This is fine for webhooks but I'm not going to let my website's backend open an HTTP connection for every event I want to send out. The decision to store the data in Elasticsearch and Cassandra is equally dubious. In my experience Elasticsearch has been a maintenance nightmare and has not been a perform any and robust reporting solution at scale.