News for hackers. Stuff that matters.

A comment on Ferret – Declarative web scraping.

Re:Ferret – Declarative web scraping by rosha on Tuesday October 02, 2018 @12:48PM (#18123362)
I am currently using https://www.npmjs.com/package/proxycrawl web scraping system that offers anonymous scraping too. But they do not have a Go library yet. Does the system support anonymous scraping too like over a proxy? I'd love to use it if I can scrape URLs with proxycrawl API like with this example: https://api.proxycrawl.com/?token=DsFuFiigAZ2Wm6U1BPh7Zw&for...
[ Parent | on HN ]
Re:Ferret – Declarative web scraping by ziflex on Tuesday October 02, 2018 @02:10PM (#18124167)
Proxy support is not in place yet. I will add it in future releases. You are welcome to do a PR :)
[ Parent | on HN ]
Re:Ferret – Declarative web scraping by megous on Tuesday October 02, 2018 @01:01PM (#18123487)
Token?
[ Parent | on HN ]
Re:Ferret – Declarative web scraping by rosha on Tuesday October 02, 2018 @01:39PM (#18123863)
I do not use tcp token, so I could share it for free, i use the api for javascript token which gives me dynamic content more.
[ Parent | on HN ]