Probably you didn't hear of scrapy https://github.com/scrapy/scrapy

+1 for scrapy, very easy to set up while remaining powerful enough to actually use!

Python has an enormous quantity of mature, multiple contributor projects.

Like the very usable Beautiful Soup: http://www.crummy.com/software/BeautifulSoup/

The author has just decided to roll their own half baked script for some reason, there are plenty of libraries to help. Python is good for scraping, I released my own library recently[1] that I think hits a sweet spot but there are many others.

Python and Ruby have their places, I wouldnt say one is clearly better than the other.

1. http://tomforb.es/scraping-websites-with-cyborg

The script is intended to be a copy/paste script, with no imports and just "Fill in the blanks" parameters. There A LOT of other options that are much better, but require more setup: http://scrapy.org/ https://pypi.python.org/pypi/spider.py

upvote - python has ridiculously good (sometimes slowish) scraping libraries, unfortunately the author has written a python wrapper over a regex which completely throws any half-sane crawling out the window.

The except: and continues is enough to cause a heartattack

I came across as aggressively flamey - but what I intended to communicate was the difference in the communities. In the Ruby community, if somebody posted a really long regex driven scraper, people would ask why it was written and recommend looking at libraries/gems instead of writing your own. In Python, it ends up on the front page of HN.

Ruby's community is very pragmatic and Python's is more scientific. Most people I'm discussing learning programming with are interested in making something, and pragmatism is far better suited to their goals.

e.g. from my perspective, as someone who uses programming as a tool, writing my own scraper is boring (other scrapers work well enough for me). From my friend's perspective, writing a scraper is a fun experiment.