News for hackers. Stuff that matters.
Re:Fast Forward - Provisioned IOPS for EBS Volumes by Smerity on Wednesday August 01, 2012 @04:57AM (#4321837)
From the previous company I worked at (which used AWS infrastructure exclusively) and many others I've spoken to, IOPS were the leading issue when it came to AWS usage. Many insane solutions were proposed (such as RAID across multiple EBS just to maintain consistent disk performance), but for most use cases the only guaranteed solution was to move everything to RAM and assume disk is either stunted or useless.

My guess is that in the coming weeks, the infrastructure of the companies I spoke to will change substantially. The SSD instances are now the goto solution if you have a server up 24/7 and (Provisioned IOPS / EBS-Optimized instances) are for when you need burst infrastructure with good disk performance.

There are gaps in the AWS armour, but those gaps have been filled in at a consistent rate.

Re:Fast Forward - Provisioned IOPS for EBS Volumes by ridruejo on Wednesday August 01, 2012 @05:29AM (#4321932)
That flexibility is the beauty of AWS in my mind. You are able to "reconfigure" your infrastructure relatively seamlessly without ever having to touch or own hardware.
Re:Fast Forward - Provisioned IOPS for EBS Volumes by abrookewood on Wednesday August 01, 2012 @07:44AM (#4322324)
I don't understand why you think utilising RAID across EBS volumes is insane. For years, if people needed more IOPs than a single drive could deliver, they utilised RAID to acheive it. How is this any different?
Re:Fast Forward - Provisioned IOPS for EBS Volumes by cagenut on Wednesday August 01, 2012 @08:37AM (#4322497)
Because EBS has astonishing variance in its response times that cause the entire raidset to have very inconsistent performance. Numbers here: http://www.mysqlperformanceblog.com/2011/02/21/death-match-e...

In order for raid-on-ebs to truly perform well you have to setup per-volume monitoring that ejects volumes from the raidset (and runs in degraded mode) when they start performing 10x worse, and then rebuilds the back on to it (or a new one) later. It becomes a rather arcane wizard work contraption of on-the-fly raid config changes, which IMHO qualifies as "insane".

Re:Fast Forward - Provisioned IOPS for EBS Volumes by rickmb on Wednesday August 01, 2012 @08:33AM (#4322483)
Even people who operate on a much smaller scale need consistent, fast performance. For them, the whole effort and cost of setting up RAID across multiple EBS instances, measuring and dropping the slowest performance constantly is completely insane compared to the disadvantages of buying or renting dedicated iron.
Re:Fast Forward - Provisioned IOPS for EBS Volumes by blo on Wednesday August 01, 2012 @05:06AM (#4321862)
Pure cost comparison:

Let's ignore the $0.10/GB-month vs. $0.125/GB-month storage pricing as it is negligible compared to IOP costs.

Standard EBS volumes average 100 IOPS with best effort bursting.

A provisioned EBS volume at 100 IOPS would cost $0.10/IOPS-month * 100 IOPS = $10/month

A standard EBS volume cost would be $0.10/1000000 IOP (an equivalent of $0.26/IOPS-month). The breakeven with the provisioned EBS occurs at 38% utilization (38 IOPS), meaning that provisioned EBS is cheaper above this point.

To match the performance of a 4 volume standard EBS RAID (400 IOPS), you would have a minimum fixed cost of $40/month/GB, vs. the current $0.40/month/4GB + variable IOP pricing.

Provisioned EBS makes sense for scenarios with sustained high IOPS utilization as you get consistent performance with possible lower total costs. For less IO-intensive loads, using a RAID of standard volumes will be more cost-effective.

Re:Fast Forward - Provisioned IOPS for EBS Volumes by jswanson on Wednesday August 01, 2012 @05:47AM (#4321999)
The way they calculate it is a little misleading.

From AWS[1]: Provisioned IOPS is charged by the amount you provision in IOPS (input/output operations per second) X the amount of days you provision for the month.

If I understand the above correctly, the estimate of $10/month would actually be:

$0.10 * 30 days * 100 IOPS = $300/month (In USEAST)

[1] http://aws.amazon.com/ebs/

Re:Fast Forward - Provisioned IOPS for EBS Volumes by RyanGWU82 on Wednesday August 01, 2012 @05:57AM (#4322024)
I think you may be mistaken -- under "Projecting Costs" that page says that it's 10 cents per provisioned IOPS month.
Re:Fast Forward - Provisioned IOPS for EBS Volumes by cperciva on Wednesday August 01, 2012 @01:39PM (#4324389)
the amount of days

They've fixed that, it now says "the percentage of days".

Re:Fast Forward - Provisioned IOPS for EBS Volumes by jswanson on Wednesday August 01, 2012 @11:02PM (#4327000)
Thank you for pointing that out, and kudos to Amazon for updating their doc. They have an easy to understand sample calculation up now as well.
Re:Fast Forward - Provisioned IOPS for EBS Volumes by bradgessler on Wednesday August 01, 2012 @04:46AM (#4321810)
Can't wait for RDS to plug into this.
Re:Fast Forward - Provisioned IOPS for EBS Volumes by jaredstenquist on Wednesday August 01, 2012 @08:49AM (#4322552)
I also wish they announced when RDS might start running off of these. In order to achieve higher IOPS i've provisioned 350GB of disk for my ~50GB database. I came across a rumor that after 300GB of space, AWS enables striping and figured it was worth the marginal extra cost.
Re:Fast Forward - Provisioned IOPS for EBS Volumes by jl6 on Wednesday August 01, 2012 @07:27AM (#4322267)
$0.10 per IOPS-month? Is that per volume or does volume size enter into the equation? I.e. would two 10GB 100IOPS volumes cost twice as much as one 20GB 100IOPS volume?
Re:Fast Forward - Provisioned IOPS for EBS Volumes by count on Wednesday August 01, 2012 @07:55AM (#4322356)
I would assume it's per-volume as 2 10GB 100IOPS volumes would be 200 IOPS, while a single 20GB volume would only be 100 IOPS.
Re:Fast Forward - Provisioned IOPS for EBS Volumes by mschalle on Wednesday August 01, 2012 @03:55AM (#4321671)
Hopefully this shuts up people always complaining about EBS...
Re:Fast Forward - Provisioned IOPS for EBS Volumes by aidos on Wednesday August 01, 2012 @04:14AM (#4321731)
Well, hopefully this helps to fix the issues with EBS that are cause for complaint....

Looks interesting, looking forward to trying it put to see how it performs.

Re:Fast Forward - Provisioned IOPS for EBS Volumes by mryan on Wednesday August 01, 2012 @04:40AM (#4321795)
I am going to be rebuilding our EC2-hosted PostgreSQL cluster soon, and will try out the provisioned IOPS feature (both with and without RAID) - happy to post benchmarks once I have some useful data.
Re:Fast Forward - Provisioned IOPS for EBS Volumes by JoachimSchipper on Wednesday August 01, 2012 @04:53AM (#4321832)
Please submit to HN!
Re:Fast Forward - Provisioned IOPS for EBS Volumes by elq on Wednesday August 01, 2012 @06:12AM (#4322057)
As someone whose team used the beta of this product - No.
Re:Fast Forward - Provisioned IOPS for EBS Volumes by bermanoid on Wednesday August 01, 2012 @10:05AM (#4322878)
As in no, you didn't consistently get within 10% of the IOPS you provisioned at their promised 99.9%? Or were there other problems?

I'd be pretty surprised if they were going live with this without being able to deliver, especially since this is likely to be very popular with people using EBS for databases under fairly consistent loads. It's one thing for I/O to suck when they're not actually promising anything, but it's quite another when you're very specifically paying for a performance guarantee...

Re:Fast Forward - Provisioned IOPS for EBS Volumes by shankys on Wednesday August 01, 2012 @07:55AM (#4322357)
I'd be really interested in hearing more details about your experience.
Re:Fast Forward - Provisioned IOPS for EBS Volumes by Andys on Wednesday August 01, 2012 @08:01AM (#4322373)
There's not really any mention of latency or what amount of concurrency is needed to hit the guaranteed level of IOPS. Is it an SSD with very low latency or just ten slow big hard disks striped together?
Re:Fast Forward - Provisioned IOPS for EBS Volumes by snorkel on Wednesday August 01, 2012 @09:58AM (#4322845)
Color me skeptical, I predict those who pay for more IOPs will howl even louder. I think by now EC2 users have learned that EBS doesn't act like a typical disk drive so treat it like one.
Re:Fast Forward - Provisioned IOPS for EBS Volumes by heretohelp on Wednesday August 01, 2012 @04:58AM (#4321844)
This still doesn't compare well to alternatives, pending benchmarks with an actual application if I directly compare it to most other IOPS benchmarks.

So...no? I mean, I don't have to complain about EBS, it's really simple, I just don't use Amazon's stack therefore I have nothing to complain about.

Now if I was somehow stuck on their stack...like from building my product against their proprietary infrastructure...then I'd be stuck on my ass and complaining for an overpriced and underperforming service.

Their recommended solution is to use their proprietary KV store.

Re:Fast Forward - Provisioned IOPS for EBS Volumes by quattrofan on Wednesday August 01, 2012 @04:24AM (#4321763)
2012 and unviewable on my mobile sad face
Re:Fast Forward - Provisioned IOPS for EBS Volumes by jordanthoms on Wednesday August 01, 2012 @05:45AM (#4321992)
Strange, it works fine in chrome for Android.
Re:Fast Forward - Provisioned IOPS for EBS Volumes by quattrofan on Thursday August 02, 2012 @08:11AM (#4328307)
Ah I should've tried that, good tip.