Re:New pagination and crawling features
by k__ on Friday February 14, 2014 @08:59PM
(#7242298)
So this is a scraper that just scrapes links?
[ Parent
| on HN ]
No. You can extract any content you want from the page (incl. text, images, links) just not meta elements or invisible ones (e.g. <title> or anything else that usually shows up in <head> for that matter)