Re:Scrapy: New Python web crawling & scraping framework (bui...
by agentbleu on Sunday December 28, 2008 @01:27PM
(#412074)
is anyone from scrapy here, i have some tech questions? is there an irc group?
[ Parent
| on HN ]
Re:Scrapy: New Python web crawling & scraping framework (bui...
by lowkey on Monday December 29, 2008 @08:24AM
(#412980)
I am looking for some community action for scrapy. It looks useful for a project I'm working on currently using BeautifulSoup but not digging the sluggish performance.
I am having trouble resolving the docs to the code. Is there an IRC, mailing list or forum?
Re:Scrapy: New Python web crawling & scraping framework (bui...
by agentbleu on Monday December 29, 2008 @01:51PM
(#413291)
the irc group on freenode has some groupies there who help. I have it installed, dont bother with debian, use a clean install of ubuntu
here are some notes to get started from a clean install (replace your own vars)
install Ubuntu 810 apt-get update apt-get install subversion adduser --home /home/bleu bleu su bleu svn co http://svn.scrapy.org/scrapy mv scrapy-trunk scrapy ls scrapy branches tags trunk sudo root as root apt-get install python-twisted apt-get install nano su bleu source ~/.bashrc pico ~/.bashrc #add this to end of file: export PYTHONPATH=/home/bleu/scrapy/trunk
python >>> import scrapy quit() scrapy/trunk/scrapy/bin/scrapy-admin.py startproject myproject