Re:Show HN: Scrape the web by injecting JavaScript into web ...
by jalfresi on Tuesday December 23, 2014 @09:46AM
(#8788275)
A very large array of mysql databases. Basically, each month we fire up a new fresh database and start streaming data into it. Were currently pulling around 700Gb a month. Our reporting tools/systems run queries across this array. Its actually not that bad speed wise (reports of over 9000 keywords over a 1 week period for top 100 positions on a per hour basis)
[ Parent
| on HN ]
There are no comments on this comment yet.