Right. The problem is that a 'bad neighbor' test is often defined as when one can only get that guaranteed minimum amount of resource.

That's the part that doesn't scale.

The amazon documentation doesn't let you pin down exactly what that guaranteed minimum is, as half the measures are things like 'IO Performance: Moderate' or 'Compute Units: 4' or aren't specified at all (like EBS performance)

Makes sense from Amazon's perspective, of course - less promises to keep, more flexibility.

Can't blame people for measuring the performance empirically, in the absence of hard guarantees. Just that produces results that happen to be wrong.

Compute units are actually a quantitative unit. One compute unit is, to the best of my recollection as I'm on mobile, the equivalent work of a specific class of 1.7GHz CPU.

Other than the "I/O Performance", I think all of their specs are pretty well defined if you're willing to dig up the appropriate docs.

I suppose it depends on whether your application depends on ephemeral disk random or sequential I/O, EBS I/O, I/O to the internet at large, cpu cache, ram bandwidth, support for AVX instructions and so on.

To be fair it's understandable why Amazon doesn't promise these features will or won't be present - it would make their already-complicated product offering even more complicated. And for a great many applications, customers won't be sensitive to details like CPU cache and disk performance.

Seems like the simplest solution is simply to pay for what you need. How many businesses are really compute-cost-bound these days?