Re:Show HN: Headless Chrome Crawler
by hartator on Thursday February 22, 2018 @10:18AM
(#16438088)
I was stuck last time I was using headless chrome when I needed to use a proxy with an username and a paasword. Headless chrome just doesn't support it. Any changes on that?
[ Parent
| on HN ]
There's a workaround - https://blog.apify.com/how-to-make-headless-chrome-and-puppe...
Thanks. Wish it was simpler. It seems overkill to have to an extra proxy in the middle with no auth to authenticate with the one with auth, just to make headless chrome working.
There's also page.authenticate, which has worked well for me.
https://github.com/GoogleChrome/puppeteer/blob/master/docs/a...
You can also use page.authenticate() for that - see a note at the bottom of the article. Also see https://github.com/GoogleChrome/puppeteer/pull/1732
Actually, we just figured out how to do this. Details here: https://bugs.chromium.org/p/chromium/issues/detail?id=741872...