I'd be really interested to know more about how the switch shaved 40% off of the response times. What was the major bottleneck? RAM? CPU? Network IO to PostgreSQL? Network IO to memcache?

Heroku is offering 2X and soon, 4X dynos; did he experiment with these?

The author also sounds rather excited about nginx serving static assets. The Rack::Cache problem on Heroku is easily solved with a CDN layer (and Heroku even recommends you do so.) (Cult Cosmetics /is/ set up with CloudFront, so the author obviously knows this.)

There are a lot of Rails architecture smells in this article, such as worrying about the 60-second boot timeout and the 30-second request timeout. [1] I have to wonder if the time invested in learning Chef and managing the architecture may have been spent making the app more performant.

Developer time is really, really expensive. Not only can you bill $100-$150 hour instead of learning how to use Chef, you could be shipping features for your own products, which can bring you passive income forever. A lot of companies are successfully running on Heroku, and it seems strange to throw everything out for the explained reasons.

[1] Specifically, adding 'require: false' to your Gemfile and only loading heavy libraries when you need them greatly helps with boot time. Ideally, you won't run any image processing libraries (like RMagick) except in background jobs, and your main app should not even need these libraries available. Same goes for PDF generation, scraping technology, or any other heavy processing.

The 30-second timeouts either need to be handled via background jobs (the author mentioned they use sidekiq) or uploading directly to S3, side-stepping the web process entirely.

I was using 2X dynos on Heroku. I did have memory issues and had to continually reduce the number of unicorns that I was running. I think CPU and disk were the major bottlenecks, but again, Heroku makes it hard to track down exactly what the cause is.

Also, we're an ecommerce site so we rely on a lot of external services (specifically for payments) where we have to run the call in the web process ... hence the issue with 30s timeouts.

> we have to run the call in the web process

I know you probably have good reason for this, but this smells like an architecture problem. Whenever you are talking to an external service it is a good idea to have your code laid out such that you can do that in the background. I understand that if you didn't plan enough architecture and maybe even product design to make that feasible from the get-go it is an expensive thing to build, but regardless of who is your hosting provider sooner or later you'll likely need to do it.

One thing I've always liked about heroku, at least in concept, is that it forces you to have good architecture. Some times it is a bit of a pain, but I think this pays off quicker than you seem to realize.

How would you do payment processing in the background without temporarily storing the payment details?

I work for a payments company.

Generally, you use JavaScript to actually tokenize the card. You then save that token in your actual application, and then use it to actually charge against.

This model works just fine with background jobs.

valid point.

I agree with the CDN comment. Please use a CDN to serve your assets no mater what you host on. It's so easy, and the performance gains to the end user are good, it should be an obvious choice.

Agree with both of you 100%, but the fact that Heroku forces this choice is a frustrating one nonetheless.

If companies think Heroku is more expensive they are failing to factor in how expensive it would be to have an employee to do all the work that Heroku takes off their plate. I work 45 - 50 hours a week developing. If I have to do devops - provisioning, deployment and system administration work - to maintain a server infrastructure there goes 15 - 20 hours! Surely my company is paying less to Heroku to manage that redundant, and at this point formulaic infrastructure than to me to do the same things that they've automated away. I can then work on our products. That seems less expensive, not more.

Once you have done the setup once and have it all automated, it's more like 1 hour per week keeping on top of upgrades, backups etc. So for many people heroku is an expensive option, even when starting out. Nothing wrong with using it, but outsourcing your infrastructure does not come cheap, compared to other options like a VPS.

VPS is much more expensive because you have to do all the work to provision (either manually or automatically) the software stack you are working with, configure every new instance either, configure your deployment and testing pipeline using a CI server and using git hooks, and keep the whole edifice up and running. The time overhead is very large, and it ends up taking over as the major cost driver.

Having done most of the steps you mention above for multile clients, I'd contend the time overhead is not very large - most things can be automated, and once that is done an extra instance is just a matter of spinning up a server and deploying - also you can scale a very long way on one or two instances when the hardware is beefy enough (e.g. this website), backup, deploy, testing can all be automated. There are definitely some situations where a service like Heroku or AWS makes a lot of sense, but it can get expensive very quickly; you're trading time to setup/manage for higher ongoing costs forever. VPS is cheaper but requires some setup. That's a trade-off each company has to evaluate individually given the talent and money they have.

Agree with most of what you say, but adding dynos increases concurrency but doesn't do anything to reduce response times. Sometimes you just need a faster box.

Or a bigger (more RAM) box, I posted this to the Ruby Rouges mailing list on tail latencies: https://gist.github.com/schneems/8849116

Concurrency is only half the battle with tail latencies. Of course dedicated CPU helps as well :)

Good stuff schneems, in my case, too little, too late.

Understood, can't be everything to every one. BTW thanks for sticking around and answering questions/engaging :)

gladly :)