Re:Crawling the web at $2 per million pages
by qeorge on Wednesday September 23, 2009 @05:46PM
(#840185)
Wow - parselets.com is amazing. Thanks for sharing.
I think 80legs can do some level of parsing now, but Parsely seems like a great way to describe the analysis for targeted crawls on predictable data. If it supported the combination you've described it would really open up some cool possibilities.
I think 80legs can do some level of parsing now, but Parsely seems like a great way to describe the analysis for targeted crawls on predictable data. If it supported the combination you've described it would really open up some cool possibilities.