News for hackers. Stuff that matters.
Re:Ask HN: How to aggregate product info from other websites by petercooper on Monday November 10, 2008 @02:43AM (#359003)
For Ruby, consider Scrubyt: http://scrubyt.org/

If you're wondering why, well, consider this script that "learns" how to scrape Google results (from one supplied example of output data):

  google_data = Scrubyt::Extractor.define do
    fetch 'http://www.google.com/ncr'
    fill_textfield 'q', 'ruby'
    submit

    link "Ruby Programming Language" do
      url "href", :type => :attribute
    end

    next_page "Next", :limit => 2
  end

  puts google_data.to_xml
Reads almost like English in the scraping part!

There are no comments on this comment yet.