News for hackers. Stuff that matters.
Re:Greplin (YC W10) AWS Benchmarks and Best Practices by epi0Bauqu on Tuesday April 26, 2011 @06:55AM (#2484666)
Thanks for putting this together. A few questions come to mind:

1) Did you try bigger instance types, i.e. large? I've heard at that level and above you get better networking performance, which theoretically might improve EBS performance?

2) Related to #1, I wonder if you've tried doing RAID0 on the ephemeral drives? You get two with the large instance, and four with xlarge.

3) Did you ever measure non-IO network EBS performance during the tests? I've always wondered whether using EBS heavily would slow down other network traffic to the device given there is only one interface.

4) How often have you yourself experienced EBS volume failure in your RAID volumes?

5) When that happens, what happens to your volumes and instance? That is, what do you use to monitor when the RAID volume degrades? Does it usually take down the instance immediately or only after some time? If it just becomes really slow does that throw alarms or does the application just become really slow?

6) Finally, what is your current procedure for dealing with a volume failure?

Well that ended up being a lot of questions. I'd really appreciate any answers/insights your or anyone else could shed on these questions. I've been reading all these posts but it feels a bit like reading tea leaves.

Re:Greplin (YC W10) AWS Benchmarks and Best Practices by jread on Tuesday April 26, 2011 @05:58PM (#2487394)
In response to question #1. I've conducted some similar benchmarking across all instance sizes and using both ephemeral and EBS with and without Raid. EBS is notably faster on larger instances. We observed roughly 2.5-3X better IO with EBS backed m2 instances compared to c1.medium. Using ephemeral Raid 0 on the cc1.4xlarge was about 6x faster.

http://blog.cloudharmony.com/2010/06/disk-io-benchmarking-in... http://blog.cloudharmony.com/2011/04/unofficial-ec2-outage-p...

Re:Greplin (YC W10) AWS Benchmarks and Best Practices by csarva on Tuesday April 26, 2011 @10:20PM (#2488123)
re: #3 - My tests show that the virtual NIC can only handle about 100k packets/sec. I don't have the results of my tests up anywhere, but Rightscale confirms this in their tests as well:

http://blog.rightscale.com/2010/04/01/benchmarking-load-bala...

So I'd say YES, other workloads would be affected during heavy EBS I/O, though I've never tested/measured this specific case.