We automate it :) There's an "inventory" of the current instances, and then using configuration management (currently Ansible) we can deploy a new version (running migrations, starting new processes and stop old ones, etc) for all or any subset of our instances with a single command.

This is why you use configuration management tools. We have a similar architecture, with dozens of deployments for customers; upgrades are just a single command that changes some configuration, and puppet does the rest. Until something goes wrong, 100 servers are no harder to upgrade than 1.

It does sound like upgrading all the customer servers would be a hassle, but that might be a feature in some domains - if customers have process / regulatory / training constraints and don't want to upgrade promptly, for instance.