Observations running 2M headless sessions

Re:Observations running 2M headless sessions by ikeboy on Monday June 04, 2018 @11:08PM (#17234684)

I scrape lots of data from services while logged in. Scraping is done with curl, but every so often cookies expire, so I use selenium to login, save cookies, then go back to curl.

[ Parent | on HN ]

Re:Observations running 2M headless sessions by smallbigfish on Monday June 04, 2018 @11:56PM (#17234869)

Isn't curl fit to also do the login part and save the cookies?

[ Parent | on HN ]

Re:Observations running 2M headless sessions by ikeboy on Tuesday June 05, 2018 @12:06AM (#17234907)

Tons of stuff get loaded during login, hard to go through each one and code the proper urls and POST params etc, quicker to write login code.

[ Parent | on HN ]