Ferret – Declarative web scraping

Re:Ferret – Declarative web scraping by the_other_guy on Tuesday October 02, 2018 @03:47PM (#18125063)

I think comments here are unfairly harsh. I really like this innovative idea of having a dedicated language. If it can see client-side rendered HTML (e.g. React, Vue, etc...), that would be a whole another level for me since I don't think this has been made before.

[ Parent | on HN ]

Re:Ferret – Declarative web scraping by ziflex on Tuesday October 02, 2018 @07:27PM (#18126542)

It can! :)

Even more - it can interact with these pages! Here is an example of use of Google Search page: https://github.com/MontFerret/ferret/blob/master/docs/exampl...

[ Parent | on HN ]

Re:Ferret – Declarative web scraping by the_other_guy on Wednesday October 03, 2018 @05:16AM (#18128704)

Wow, thank you. Now I am absolutely in love with your tool!

[ Parent | on HN ]

Re:Ferret – Declarative web scraping by ziflex on Wednesday October 03, 2018 @08:02AM (#18129361)

Great! ^_^

[ Parent | on HN ]

Re:Ferret – Declarative web scraping by spaceribs on Tuesday October 02, 2018 @04:38PM (#18125509)

I feel the same here, architecturally I've never liked scrapers which are tightly coupled within a service. Websites are prone to sudden change and it seems unsustainable to redeploy each time a selector changes, it's pretty innovative to leverage the power of *QL for this sort of problem.

[ Parent | on HN ]

Re:Ferret – Declarative web scraping by ziflex on Tuesday October 02, 2018 @07:28PM (#18126549)

Yes! This is one of the reasons why I wanted to be able making these changes without redeploying the whole thing!

[ Parent | on HN ]