Re:Ask YC: Any ideas about intelligent crawlers :)
by marketer on Tuesday January 08, 2008 @04:20PM
(#96122)
I think this is the best way to go. There's no reason that you should be scraping HTML from sites, when there might be a nice xml feed available. For instance, pricegrabber will only index your site if they have prices in XML.
[ Parent
| on HN ]
There are no comments on this comment yet.