News for hackers. Stuff that matters.
Re:Beer and Concurrent HTTP Pipelines by hrbrmstr on Sunday October 30, 2016 @07:14AM (#12829177)
Rather than emoticon-ing away your responsibility in a potential site DoS, perhaps read up on responsible crawling - https://webarchive.jira.com/wiki/display/Heritrix/Responsibl... + http://blog.mischel.com/2011/12/20/writing-a-web-crawler-pol... . You had no need for either unlimited or even 50 parallel crawl tasks. And, if your intent is anything but personal use, you shld prbly (re)read https://www.brewtoad.com/legal since you just got their attention in a pretty big way.