News for hackers. Stuff that matters.

A comment on IBM Watson Acquires Technology from Blekko. Parent comment

Re:IBM Watson Acquires Technology from Blekko by metronius on Monday March 30, 2015 @04:30AM (#9288271)
If you would crawled web using Foreach(links) { get(link)} you would end up crawling useless automatically generated webpages.
[ Parent | on HN ]
Re:IBM Watson Acquires Technology from Blekko by psykovsky on Monday March 30, 2015 @07:18AM (#9288768)
You say that like it doesn't happen all the time. googlebot included.
[ Parent | on HN ]