News for hackers. Stuff that matters.
Re:Show HN: BBC Good Food Scraper in Go by imjacobclark on Thursday June 23, 2016 @03:47AM (#11959438)
It seems with all these website scrapers, we have forgotten about the Internet Archive. http://archive.org/web/
Re:Show HN: BBC Good Food Scraper in Go by anc84 on Thursday June 23, 2016 @07:42AM (#11960225)
As awesome as the Internet Archive's Wayback Machine is, it is still under central control. Worse, as they (reasonably) abide to robots.txt rules, the BBC could easily block access to pages they removed at their end in the archive as well. If you care about something, you need to fully "own" it.
Re:Show HN: BBC Good Food Scraper in Go by johtso on Thursday June 23, 2016 @05:22AM (#11959730)