Re:Crawling the web at $2 per million pages
by jdrock on Thursday September 24, 2009 @01:31AM
(#840663)
We're not using EC2. AWS doesn't and can't scale to our size, actually :)
Side note: AWS guys have asked if we use them. We said "No, we'd be losing money then and wouldn't be able to scale to our size with you."
The reason why they didn't want it to be too expensive (ie. 5 times as expensive) is (1) competitors can easily equilibrate and steal market share if their idea works b/c of the economic inefficiencies in their pricing model and (2) this game plan is more a game of dependence rather than up front profits, so it makes sense to take very little profit up front to get user traction.