In cases like this, where both the solutions are software, the reason the company is relying on a progressively Matryoshka'ed system built on top of the original, "dumb" solution, is usually that they value developer time (to build a better solution) greatly over ops-staff time (to implement the infrastructure required to scale the dumb solution) + additional hardware costs (for all the overhead the dumb solution's method of scaling introduces.)
But even that doesn't explain why organizations refuse to switch from bailing-wire scripts to a pre-built, commonly-available infrastructure-component better solution. In such cases, both staying and switching are ops costs.
The costs of staying are known, because they’re the ones you’ve already been paying for a while. Switching to a new system is inherently risky as there’s always a significant chance you haven’t correctly identified all of the requirements from the existing system.
This is a dangerous assumption. It seems at least equally likely that they are just responding to the incentives the system presents them. In other words, when you see a chain gang, you're seeing an organization that views hardware as expensive and people as cheap.