Just in time. That XPath example is really useful for me.

I didn't finished the article yet, but I have one question. Is there ability in Scrapy to append new data after initial scrapping?

For example site at the moment of scrapping:

  Article 1  
  Article 2

Then we run scrapy again after some time and site will look like this:

  Article 1  
  Article 2  
  Article 3  
  Article 4

Can it just append article 3 and 4 to file/db table and stop scraping?