News for hackers. Stuff that matters.
Re:Show HN: Scrape the web by injecting JavaScript into web ... by jalfresi on Tuesday December 23, 2014 @09:46AM (#8788275)
A very large array of mysql databases. Basically, each month we fire up a new fresh database and start streaming data into it. Were currently pulling around 700Gb a month. Our reporting tools/systems run queries across this array. Its actually not that bad speed wise (reports of over 9000 keywords over a 1 week period for top 100 positions on a per hour basis)

There are no comments on this comment yet.