I know I was vague, but we really are CPU-bound. We do some fairly heavy set and bitwise operations. Our smallest unit of work is about 3x-5x slower on AWS (while costing about 4x more than alternatives). This can't be further parallelized without a rewrite. The problem is compounded by the negative impact this has on concurrency - which can be solved by adding more machines, but that just makes AWS that much more expensive.

A different, less important system, does image manipulation. This is also very CPU-sensitive.

Interesting

Maybe I need to rephrase what I said: AWS instances are very bad at CPU, but the flexibility to add more instances and different instance types can compensate for that

Yes, maybe you can try Linode or Rackspace to compare their CPU performance

Really? I was just using aws to run some load testing bots against our game servers and found the CPUs even in High CPU instances pitiful for the price.

The worst part about it is that the performance is so variable between instances. Some could comfortably run 1300 bots while others were slowing down so much after 750 that they started dropping bots.

Because of that, it's easier to just run 750 on each which is wasting resources.

My desktop PC can easily run 4000.

Yes, desktop PCs are much faster than an EC2 instance

Maybe Linode or Rackspace (or other solution) is better for your case.

Still, apparently for your case you can just add more servers according to demand, which is the advantage of EC2

CPU is a limiting factor for a lot of applications. And EC2's offerings are paltry until you get into the really expensive VMs. But it's not just CPU, it's the overall performance. Engineers want to run on hardware for many more reasons than CPU. And for most apps it's the rational choice. After all, you can build your own data center for the price you pay to rent some virtual machines.

tl;dr: With cloud services you pay thousands a month for _renting software_.