News for hackers. Stuff that matters.
Re:Show HN: Headless Chrome Crawler by hartator on Thursday February 22, 2018 @10:18AM (#16438088)
I was stuck last time I was using headless chrome when I needed to use a proxy with an username and a paasword. Headless chrome just doesn't support it. Any changes on that?
Re:Show HN: Headless Chrome Crawler by jancurn on Thursday February 22, 2018 @10:34AM (#16438208)
Re:Show HN: Headless Chrome Crawler by hartator on Thursday February 22, 2018 @10:42AM (#16438279)
Thanks. Wish it was simpler. It seems overkill to have to an extra proxy in the middle with no auth to authenticate with the one with auth, just to make headless chrome working.
Re:Show HN: Headless Chrome Crawler by timstapl on Thursday February 22, 2018 @11:00AM (#16438466)
There's also page.authenticate, which has worked well for me. https://github.com/GoogleChrome/puppeteer/blob/master/docs/a...
Re:Show HN: Headless Chrome Crawler by jancurn on Thursday February 22, 2018 @11:00AM (#16438469)
You can also use page.authenticate() for that - see a note at the bottom of the article. Also see https://github.com/GoogleChrome/puppeteer/pull/1732
Re:Show HN: Headless Chrome Crawler by transreal on Thursday February 22, 2018 @05:58PM (#16442354)
Actually, we just figured out how to do this. Details here: https://bugs.chromium.org/p/chromium/issues/detail?id=741872...
Re:Show HN: Headless Chrome Crawler by hartator on Thursday February 22, 2018 @06:08PM (#16442428)
Awesome. I need to figure out a way to make it work with our Ruby code, but it shouldn't be that hard. Thanks.