Tuning cluster rebalance settings can be... challenging. I've done some bad, bad things with a cluster where it would effectively balance things back and forth - forever.

Fiddling with cluster.routing.allocation.balance.* can really paint you into a corner quickly.