News for hackers. Stuff that matters.
Re:Only Google is really allowed to crawl the web by indymike on Friday March 26, 2021 @03:44PM (#26596117)
I used to run a fairly large job ad scraping operation. Our scraped data was used by many US state and federal job sites. "Scraping" is just using software to load a page and extracting content. "Crawling" is just load a page, find hyperlinks (hmm... a kind of content), and then crawling those links. Crawling is just a kind of scraping.

There are no comments on this comment yet.