News for hackers. Stuff that matters.

A comment on ElasticSearch, logstash and kibana. Our guide to big data with the ELK stack. Parent comment

Re:ElasticSearch, logstash and kibana. Our guide to big data... by andyidsinga on Wednesday November 05, 2014 @04:58PM (#8564527)
ive thought about this too when considering an architecture like the one described ..seems as though storing in S3 is a good first stop for the data as it takes very little care and feeding for a massive amount of data.

then, depending on your query and data processing needs you can look to other tools such as elastic search, or elastic map reduce or xyz db.

see also : http://petewarden.com/2010/10/01/how-i-ended-up-using-s3-as-...

[ Parent | on HN ]
Re:ElasticSearch, logstash and kibana. Our guide to big data... by pcreux on Wednesday November 05, 2014 @05:37PM (#8564686)
S3 is quite limited when it comes to listing a subset of objects as scope using prefixes. So can fetch all objects starting with "2014/10/10" but you can't fetch the last 30 created objects for instance.
[ Parent | on HN ]