News for hackers. Stuff that matters.

A comment on Show HN: WrapAPI v2 – Build APIs, scrapers, bots on any website.

Re:Show HN: WrapAPI v2 – Build APIs, scrapers, bots on any w... by salimmadjd on Friday April 21, 2017 @02:34PM (#14168083)
What if the content you want to scrape->API is behind a login gate? Is there an option of authentication?
[ Parent | on HN ]
Re:Show HN: WrapAPI v2 – Build APIs, scrapers, bots on any w... by wrapapi on Friday April 21, 2017 @02:38PM (#14168128)
You can actually write an API endpoint that'll retrieve a state token (which includes the cookies). For one example, you can use view our Hacker News login endpoint at https://wrapapi.com/#/view/phsource/hackernews/login/latest

That endpoint will then emit a state token, which includes the session cookies. You can feed that state token into your next request and it'll authenticate you

[ Parent | on HN ]
Re:Show HN: WrapAPI v2 – Build APIs, scrapers, bots on any w... by webninja on Friday April 21, 2017 @06:36PM (#14169965)
Yes, you can get the content you want even if it's behind a login page! Expect to create 2 APIs, one for logging in and another for getting the content. An example is provided on their homepage: https://wrapapi.com/v2#/caseStudies/cj
[ Parent | on HN ]