Re:A Little Web Spider I Wrote Last Night (code included)
by jerf on Thursday March 12, 2009 @06:02PM
(#514175)
Here's a little web spider I just wrote: "wget -r".
I haven't got a problem with people writing their own quickie scripts, but they aren't really worth putting online without some other compelling reason.
Re:A Little Web Spider I Wrote Last Night (code included)
by timf on Thursday March 12, 2009 @05:07PM
(#514096)
It's hard to tell if you're after learning or something to use etc., but if you haven't seen it yet, check out http://scrapy.org/ for ideas (or even something to use if that is what you are after).
http://www.foo.be/docs/tpj/issues/vol4_3/tpj0403-0013.html
I haven't got a problem with people writing their own quickie scripts, but they aren't really worth putting online without some other compelling reason.
http://github.com/michaelmelanson/spider