jeremybmerrill

Developer here. Sorry, all the HN traffic overwhelmed my t2.micro. Had to upgrade a li'l bit.

Here it is: https://www.crossing.us/intersection/06110498932921_11049893...

you've got me beat there, doppelganger!

I love the idea of putting the bus tracker inside a model bus. Do you mind if I steal that idea?

who needs practical uses?

Thanks, will check it out.

Thanks, Dan. I'll check that out, especially for good ideas on how to solve things I haven't solved yet. Spidering and scraping seem to be very related, but not quite the same -- and I admittedly know nothing about spidering.

I had never seen this before. I will check it out, at least for inspiration.

I'd say "scraping" is a little more focused on extracting data from specific pages as opposed to ALL pages as in "spidering", but the two are certainly cousins if not siblings. Anemone would probably be good at the same sorts of tasks Upton is designed for (i.e. scraping data contained on multiple pages).

Thanks, I hope you like it.

Nope. Sorry. :)

While I'm sure this would be possible, I think a node.js scraper would probably be a better fit for that sort of a project.

AFAICT, YQL can only handle scraping individual pages that way.

Upton can scrape a whole set of pages. If you have a page that lists the pages you're interested in; suppose you're interested in HN commenters on front page posts, you could specify the front page URL and a selector for links to comment pages, and Upton would automatically scrape those pages and return them to you.

Upton could even write the commenter names to a CSV for you with just a filename and a CSS selector/XPath expression.

It's not stuff you couldn't do with YQL or Python/BeautifulSoup. But it's stuff that I didn't want to have to write over and over each time I wrote a new scraper.