You're right that most of the time you don't need to use JavaScript.

But look at Google Groups for example - there is an infinite scroll to get all the topics, posts are also loaded dynamically, so you have to wait some time to get them.

In the SFO flights example you have to deal with pagination also using JavaScript.

We wanted to build a powerful tool which can crawl and scrape almost any website out there. It's slower, but you can use bench of our nodes to do it in parallel.

I agree that the Google Groups example is much simpler when using PhantomJS, but I would argue that it would be an outlier.

The SFO flights example is actually heavily over engineered, from quickly glancing over the XHR tab in Chrome Network tools it was pretty obvious that all of the data is actually located in this very nice JSON blob http://www.flysfo.com/flightprocessing/fullFlightData.txt

(I assume that the SF Flight Info was just meant as an example for the platform and as such the fact that it's already a JSON blob was just ignored for the sake of the example)

I've written some projects which use phantomjs; the primary motivation for me has been the desire to look at the web in general, rather than specific sites I'm scraping data off, and having the ability to see what their javascript does.

It's OK when you only have to crawl one or two websites, sure, manually analyze the js and write minimal DOM parsing routes would do.

But how about hundreds or thousands of websites to crawl? Or do you prefer just use phantomjs write static extraction rules.

If you limit to just that, then there's no benefit over 10 lines of Ruby and Nokogiri.