What are you doing with 40 racks of hardware? Serious question. That's A LOT of computers! I assume you mean 40 42 racks? That's like, 800-1000 computers depending on whether you do 1 vs. 2u machines, or use blades, how dense the switching is, how full the racks are, etc.
I was in charge of tech ops for a billion-device scale analytics company and we ran 100-200 VM instances on EC2. I can't imagine needing hundreds of bare-metal instances. I even lived with the guys running Firebase. Before they got acquired by Google, I think they had, maybe a dozen or two bare-metal instances at Softlayer.
What about the salary of the person who maintains all of that? The cost of spare parts? The cost of downtime when hardware breaks?
And, yes, all-inclusive, AWS is 1.5x-10x more expensive than commodity hardware, depending on how poorly optimized AWS's hardware choices were for the particular workload and the commercial datacenter market at the time.
> What about the salary of the person
In general, I've found the need for the quantity of "person", bizarrely, exaggerated.
> The cost of spare parts?
Included, and it's low. Is this another aspect that's exaggerated?
> The cost of downtime when hardware breaks?
This is identical to the cost of downtime when AWS's hardware breaks.
I've worked on 40-rack build outs using supermicro without any special pricing that beat the cost of AWS for an equivalent number of reserved instances.