Re:Lessons Learned from a Year of Elasticsearch in Production
by warriormonk5 on Monday March 14, 2016 @02:37PM
(#11284594)
I wonder what size of cluster they are using.
I have two clusters running.
Largest one is an ELK stack. 7 nodes (soon to be 8, we're logging too much :) ), no dedicated master nodes, ~3TBs of data (1.5TB data x 2 for the back up). Need to make sure they are split appropriately into availability zones in case you lose a cloud zone. Need to have the right # of minimum master nodes otherwise you can hit a split brain issue. 500GB SSD, 4vCPUs, 26gigs memory. It eats ~5 gigs an hour peak load without breaking a sweat. Can basically take any heavy query people have thrown at it kibana wise.
I have two clusters running.
Largest one is an ELK stack. 7 nodes (soon to be 8, we're logging too much :) ), no dedicated master nodes, ~3TBs of data (1.5TB data x 2 for the back up). Need to make sure they are split appropriately into availability zones in case you lose a cloud zone. Need to have the right # of minimum master nodes otherwise you can hit a split brain issue. 500GB SSD, 4vCPUs, 26gigs memory. It eats ~5 gigs an hour peak load without breaking a sweat. Can basically take any heavy query people have thrown at it kibana wise.