News for hackers. Stuff that matters.
Re:80legs Web-Scale Apps Competition by jdrock on Monday October 19, 2009 @02:10PM (#890587)
The 50,000+ computers come from a distributed grid computing system that's similar to a commercial opt-in SETI@home.

Because of that, we don't know exact specs on the computers, but here are the limitations that 80Apps have: http://80legs.pbworks.com/80Apps#LimitationsonYourCode

Re:80legs Web-Scale Apps Competition by vicaya on Monday October 19, 2009 @05:18PM (#891051)
Well, do you have something like this?:

http://setiathome.berkeley.edu/top_hosts.php

BTW, how do you ensure somebody is not submitting a malware (intentional or not) that can DDoS someone by crawling the same host at pretty much the same time?

Re:80legs Web-Scale Apps Competition by jdrock on Monday October 19, 2009 @06:14PM (#891204)
I know Plura uses IP information to prevent bandwidth overages, but I don't think they have a list of top hosts.

80legs enforces throttles and rate limits on the sites that are crawled. We try to make a best guess at a rate limit for each site. Sometimes that best guess is too fast, so we take corrective action when we're notified by the webmaster. Users can't override the crawling rates.