I really badly do not understand what EC2 gives you that a dedicated box doesn't. Yes, if you use all the other services, then there's a value but then again for example SQS is a particularly shitty queue.

It's not ec2, it's the ecosystem.

DDOS, multi-path, multi-az, dr, snapshots, hardware redundancy, elasticity, freedom from datacenter contracts, fiber contracts, driving to rack stuff, jammed fingers, staff costs, firmware bugs, bad hardware batches, cdn, compliance costs. I have more but I'll stop there.

It doesn't make sense at very small or large scales but it captures a hell of a lot of the middle.

Most of what you described is colo and not dedicated boxes...

And honestly, even colo doesn't take that much time unless you're scaling up like crazy. Once we racked our machines, we almost never had to touch them. You can get a LOT of resources in a single box for $10k/server these days.

> SQS is a particularly shitty queue.

Do you have any other recommendations for queue systems that can handle a million messages per minute (our use case)?

A million messages per minute ? How's that a problem? Please see http://bravenewgeek.com/dissecting-message-queues/

Almost all of the systems tested hit 17k / s (aside from NSQ) -- on a laptop. NATS (gnatsd) hits 0.2M per second, that's a magnitude above what you need and that's a brokered system, if your task fits with a brokerless one then nanomsg is yet another magnitude. I do not particularly see where the problem really is?

Our messages range from a few KB to a few MB (we store a pointer to S3 for the large ones) but that look pretty interesting.

Thanks

It gives you dev ops out of the box. For small projects it has its merits. For most developers tinkering with the server isn't high in their priority list. I'm with you on dedicated but if I were to run an asynchronous crawler I'd rather go with AWS and save me the trouble of setting up the proper environment.