If you halve the memory and CPU usage of a given service (that's not IO bound), have you not improved it's vertical scalability?

Not if those things aren't the bottleneck. Scalability is such a buzzword. The correct term is "performance" or "overhead". Scalability matters when you're talking about an entire system that you're deploying on many machines -- the problem space and technical discussion is different to fixing a performance issue in Python.

David from Sentry here. I've got a long history of talking about scale in the python/web ecosystems. I'll echo what some others have said: scale is fundamentally a business metric.

If we can do more with less machines it's no different than doing more with more machines. We added triple our infrastructure to handle the Python load while we resolved this CPU issue, and that wasn't "scaling" (literally, it causes other scale concerns).

Fixing the root cause let's us drop all of those new machines as well as some older ones. The scalability of the system has greatly increased because of this and other factors -- primarily that many or all systems aren't actually "horizontally" scaleable.