Re:Crawling the web at $2 per million pages
by Flemlord on Wednesday September 23, 2009 @05:02PM
(#840119)
I wonder how they arrived at their $2 per million pricing model. Why not $10 per million? Either one sounds ridiculously inexpensive.
[ Parent
| on HN ]
Re:Crawling the web at $2 per million pages
by charlesju on Wednesday September 23, 2009 @05:05PM
(#840126)
They're probably using EC2s and then it's a function of how many pages can be scrapped per EC2 compute hour and then taking a premium on top of that.
The reason why they didn't want it to be too expensive (ie. 5 times as expensive) is (1) competitors can easily equilibrate and steal market share if their idea works b/c of the economic inefficiencies in their pricing model and (2) this game plan is more a game of dependence rather than up front profits, so it makes sense to take very little profit up front to get user traction.
Re:Crawling the web at $2 per million pages
by jdrock on Thursday September 24, 2009 @01:31AM
(#840663)
We're not using EC2. AWS doesn't and can't scale to our size, actually :)
Side note: AWS guys have asked if we use them. We said "No, we'd be losing money then and wouldn't be able to scale to our size with you."