News for hackers. Stuff that matters.
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by falcolas on Tuesday January 12, 2016 @10:22AM (#10887879)
Hoo boy. Here we go. The problem with AWS reps is that they only see everything as working perfectly, with no possibility for downtime of their services.

RDS is great, but only to a certain level. You'll still need to pull it off RDS once you reach that service's capacity (much sooner than their 10m user mark). They also keep pushing Aurora, but without telling us what the tradeoffs are for the high availability. Based on the responses so far (MySQL backed by InnoDB), it appears to be based on a technology similar to Galara, which has a lot of caveats for its use, especially with multiple writers.

Don't depend on Elastic Scaling for high availability - when an AZ is having issues, the AWS API will either be down or swamped, so you want to have at least 50% extra capacity at all times, if you want high availability.

Using their scaling numbers, your costs start spiking at 10 users. Realistically, with intelligent caching (even something as simple as Nginx caching), you can easily support several thousand users just fine with a t2 style instance, either a small or micro. Splitting services onto different hosts not only increases your hosting costs, it increases the workload on your developers/admins and likeliness of failure.

DR: Don't wait until you have over a thousand users to have multiple instances in different AZs. The cost of duplicating a t2.small across an AZ is small compared to lost users or sales.

Automation: Be prepared for vendor lockin if you use Amazon's solutions. Also be prepared for their APIs being unavailable during times of high load or during AZ failures.

> Lambda [...] We’ve done away with EC2. It scales out for you and there’s no OS to manage.

The biggest problem with Lambda right now are the huge latency costs with cold lambda instances. You'll get a pretty good 95% percentile response times, but that other 5% will be off-the-chart bad.

In summary, AWS has a lot of great toys, and can absolutely be used for scaling up to silly levels. However, most who have done this degree of scaling do not do so using AWS tools.

Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by LoSboccacc on Tuesday January 12, 2016 @11:26AM (#10888285)
> Realistically, with intelligent caching (even something as simple as Nginx caching), you can easily support several thousand users just fine with a t2 style instance

agreed, the article approach to scalability is to throw silly amounts of money at the problem, instead of going for an architecture to squeeze first every bit of performance out of the app. true this approach is pretty simple and works for any kind of application, but the RDS will hit connections cap quite fast if on just throws instances at the problem.

edit: yep, just noticed this comes from a Amazon Web Services Solutions Architect, of course the solution is to throw money at them

Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by falcolas on Tuesday January 12, 2016 @01:28PM (#10889274)
> of course the solution is to throw money at them

Yup. They put out a white paper at one point on surviving DDOS attacks on AWS which amounted to "out-scale the attack". AKA the Wallet based DDOS.

Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by semi-extrinsic on Tuesday January 12, 2016 @05:41PM (#10890971)
> you can easily support several thousand users just fine with a t2 style instance

Yep. I've recently load tested (with Locust) a Flask/uWSGI/Nginx webapp I built that does Pandas DataFrame queries based on user input and serves data computed from the query result. I put a bit of effort into profiling and optimizing the Python code^1, and I do caching in uWSGI. Running on the equivalent of a single t2.small instance, it can handle about 70,000 requests per hour, which I figure is the equivalent of a few thousand simultaneous users^2. For just serving a dynamic webpage from Flask it can handle almost a million requests per hour.

^1 (Surprisingly, a Pandas DataFrame lookup like `df[df.alpha == input]` can be almost an order of magnitude faster if you replace `df.alpha` with `df.alpha.values`.)

^2 (The data it serves is input for simulation codes which take hours to run on the user's hardware, so 30 lookups per hour is probably more than a typical user would do.)

Edit: asterisk doesn't work as a footnote symbol here...

Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by late2part on Tuesday January 12, 2016 @10:35AM (#10887963)
Agreed - they are a great solution for small teams that are growing fast and don't have predictability. But, once you have some level of predictability and scale, it makes sense to move off to something much higher performance and lower cost. Until you become a decrepit Fortune 50 company and can't manage an IT department due to bloat, and it's cheaper to outsource.
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by rjsamson on Tuesday January 12, 2016 @11:08AM (#10888155)
Curious what you see out there that's higher performance and lower cost than AWS? In my experience it's been a great fit for small apps all the way up to large complicated applications at scale - and once your infrastructure is large enough you're buying reserved instances anyway at anywhere between a 33% and 70% discount.
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by vidarh on Tuesday January 12, 2016 @11:26AM (#10888281)
You can beat AWS on cost with pretty much any hosting provider (with some exceptions - e.g. Rackspace seems almost proud to be expensive). The 33% to 70% "discount" doesn't mean much when you then tie yourself into long term costs that are far more limiting than most manage hosting providers - so much for benefits of being able to scale up and down.

What really kills you on AWS are the insane bandwidth prices. Buying bandwidth elsewhere is often so much cheaper than AWS that the difference in bandwidth costs alone more than finances the servers.

Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by semerda on Tuesday January 12, 2016 @11:35AM (#10888366)
How is Netflix able to manage this so effectively and still serve ~30% of US traffic off AWS?

I've heard the non-AWS folks talk of these vendor lock ins or long term costs but aren't those irrelevant in 2016+? eg. microservices to reduce the issue of vendor lock in and long term costs on infrastructure that goes out of date every 2-3 years is a poor planning indicator no?

Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by vidarh on Tuesday January 12, 2016 @11:51AM (#10888472)
I can guarantee you that Netflix are not paying anything remotely like the advertised rates for EC2.

I know first hand the kind of discounts some companies much, much smaller than Netflix can get, and they are steep. EC2 is still expensive then too, but if you pay, say, a million a year to Amazon without massive discounts, you've not done your job when negotiating.

But yes, someone with the leverage Netflix has will be paying relatively reasonable rates for EC2 services. But pretty much nobody else has the leverage Netflix has.

> I've heard the non-AWS folks talk of these vendor lock ins or long term costs but aren't those irrelevant in 2016+?

Paying far above market rates is never going to be irrelevant, because if you pay above market and your competitor doesn't, chances are they'll have you for breakfast thanks to better margins.

Why in the world would you agree to pay above market rates to get locked in for 1-3 years when you can pay less on a month-by-month contract?

Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by secabeen on Tuesday January 12, 2016 @02:35PM (#10889753)
Netflix could even be paying less than cost, as a loss-leader for AWS.
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by semerda on Tuesday January 12, 2016 @04:12PM (#10890430)
Feels like AWS is less of a vendor lock than building it inhouse. Doing it all inhouse has a high upfront cost that must be realized over X years irrelevant of the outcome. On the other hand if one implemented a microservices architecture, moving off AWS month-to-month service to another provider is far easier. Did I miss something?
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by manigandham on Tuesday January 12, 2016 @07:04PM (#10891378)
How is microservices related here? They're built in-house too. It's still just services/apps/code that has to run somewhere.

You can run it on AWS or somewhere else but moving is always a problem regardless.

Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by late2part on Tuesday January 12, 2016 @11:06PM (#10892264)
There are no month-to-month costs with Amazon that I'm aware of. There are hour by hour, and 12 month and 36 month commitments.
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by pvg on Tuesday January 12, 2016 @12:47PM (#10888928)
Netflix does not stream content from AWS.
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by toomuchtodo on Tuesday January 12, 2016 @12:48PM (#10888941)
+1. Netflix.com is only the control plane, all content is served from CDNs.
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by late2part on Tuesday January 12, 2016 @11:08PM (#10892274)
The majority are all of Netflix's CDN traffic comes from their own CDN that they do not run on Amazon.

In fact, they don't even use the same hardware or software.

https://openconnect.netflix.com/software/

Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by Spooky23 on Tuesday January 12, 2016 @03:00PM (#10889919)
Keep in mind that Amazon (and others) uses the "roach motel" model for networking. Easy to check in, not so easy to check out.

When we looked at S3 for some archiving use cases, that came up as a risk -- if strategically it made more sense for us to adopt Google, Microsoft, etc, we would need to negotiate significant concessions from a new vendor to transition away from Amazon or take a hit during that period. You always need to plan for the exit!

You'll have similar issues on-premises (ie. dealing with EMC/etc), but many people forget that cloud providers have their own gotchas too.

Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by noir_lord on Tuesday January 12, 2016 @11:47AM (#10888437)
I suspect Netflix is paying something a lot closer to AWS cost price than any of us will get.

TBH The cost of AWS isn't what concerns me so much as the massive vendor lock-in.

Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by grepory on Tuesday January 12, 2016 @03:31PM (#10890129)
Vendor lock-in is an unavoidable cost of doing business. Even if you build literally everything yourself, which you shouldn't, you still have resources, processes, apis, automation, expertise amassed around a specific set of operating constraints.

Not only that, but if you invest significantly in any single technology, migrating to another technology is always going to be an extreme effort. Having led migrations from datacenters to AWS, AWS to Digital Ocean, RabbitMQ to NSQ to SNS+SQS, etc., I can say at this point that I do not believe in vendor lock-in as a legitimate reason to disqualify any particular solution.

Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by Spooky23 on Tuesday January 12, 2016 @02:44PM (#10889807)
In my mind, it's like leasing a car. Leasing is better for your cash flow, but buying is usually a lower total cost.

Outside of large volume S3, it's pretty trivial to beat AWS costs, assuming you have the human capability. S3 is a little different, as the capital investment required to host petabytes of data is very high, and Amazon's economy of scale is pretty compelling.

For most anything else, dedicated boxes at a colo or your own datacenter should be cheaper, assuming you have the people around to do it, etc

Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by gshx on Tuesday January 12, 2016 @12:43PM (#10888899)
The other problem with Lambda is that you cannot keep persistent connections in a connection pool. It is after all, designed for statelessness. This can be considerable cost for doing calls to other business services (http connection pools) or infra services like databases that all maintain persistent connections.
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by wsh91 on Tuesday January 12, 2016 @02:22PM (#10889665)
This isn't true. I run a Lambda right now that queries a Cassandra connection pool at high volume. In Java, at least, you set up your resources in a static initializer block, as this alludes to. http://docs.aws.amazon.com/lambda/latest/dg/best-practices.h... Problem solved.
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by falcolas on Tuesday January 12, 2016 @01:29PM (#10889281)
Absolutely. The overhead of re-establishing a secure DB connection for every request is hardly trivial.
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by wsh91 on Tuesday January 12, 2016 @02:23PM (#10889681)
It would be, if it were necessary, but it's not. (Static initializers or default constructors in Java, for example.)
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by falcolas on Wednesday January 13, 2016 @04:59PM (#10897896)
Question then: How do you omit the overhead of setting up a new socket and all of the SSL handshakes? I'm not concerned about the Java overhead associated with new connections, I'm concerned with the raw connectivity/handshake overhead required with new connections to the DB.
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by wsh91 on Monday January 18, 2016 @02:38PM (#10926385)
It happens once, on initialization. :) The first execution takes anywhere from 50-70 seconds, for sure, but reusing the connection afterwards means subsequent ones don't have to deal with it (100-200 ms a pop). (Does that make sense?)
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by ec109685 on Tuesday January 19, 2016 @07:03AM (#10930138)
50 seconds?
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by ch8230 on Tuesday January 12, 2016 @11:34AM (#10888362)
Agreed on Lambda latency costs. I've used it to process API calls and I noticed it can add almost half a sec to the response or sometimes even longer.
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by cddotdotslash on Tuesday January 12, 2016 @12:12PM (#10888657)
This is a bit of a hack workaround, but all you need to do is have the function run at least every ten minutes. So, using the scheduled task feature, just kick off an event every ten minutes that invokes the function with a custom event that you can respond to instantly within the event handler (to minimize costs). Once you set that up, the function will never scale down and you'll always get hot boot times for just a few pennies extra per month.
Re:A Beginner's Guide to Scaling to 11M+ Users on Amazon's AWS by djhworld on Tuesday January 12, 2016 @04:40PM (#10890603)
tbh if you're that concerned about the 5% of response times being affected by cold Lambdas, then maybe lambda isn't really the solution to the problem you are trying to tackle.