News for hackers. Stuff that matters.
Re:AWS Elasticsearch: a fundamentally-flawed offering by cloakandswagger on Friday October 11, 2019 @07:43PM (#21229413)
The whole shard rebalancing problem is solved if you have an acceptable disaster recovery solution in place, which I'm assuming the author doesn't if they're complaining about have to keep around the raw copy of their data to be indexed.

My team has an automated workflow that runs once a week. It creates a new cluster, re-indexes from source, starts taking customer traffic and then deletes the old cluster. The shards stay balanced, and we can recover from a total cluster failure within about 6 hours.

Re:AWS Elasticsearch: a fundamentally-flawed offering by Esthrowaway123 on Friday October 11, 2019 @09:43PM (#21229995)
Yes, you should have a DR scenario.

No, you should not need to use it because an aws product is a piece of shit.

Re:AWS Elasticsearch: a fundamentally-flawed offering by DominoTree on Friday October 11, 2019 @07:57PM (#21229464)
Can I ask how large your clusters are and how many records you’re reindexing? I haven’t been able to scale this to a cluster taking in several TB a day