News for hackers. Stuff that matters.

A comment on Avoiding bot detection: How to scrape the web without getting blocked?. Parent comment

Re:Avoiding bot detection: How to scrape the web without get... by Mister_Snuggles on Sunday October 31, 2021 @10:48PM (#29062577)
I had never heard of this, but it looks like a reasonable option. My go-to for this type of thing would be Python+Selenium+Firefox, but only due to familiarity with those.
[ Parent | on HN ]
Re:Avoiding bot detection: How to scrape the web without get... by at_compile_time on Monday November 01, 2021 @02:36AM (#29063658)
I'm glad that I'm not the only one not using Selenium for its stated purpose.
[ Parent | on HN ]
Re:Avoiding bot detection: How to scrape the web without get... by shapefrog on Monday November 01, 2021 @05:22AM (#29064516)
Is there another - stated purpose - that I missed?

Primarily it is for automating web applications for testing purposes

Here was I thinking it was just a tool to scrape all the js I am too lazy to reverse engineer.

[ Parent | on HN ]
Re:Avoiding bot detection: How to scrape the web without get... by cebert on Monday November 01, 2021 @06:12AM (#29064812)
Playwright is easy to get started with. The even tools that allow you to record your browser actions and covert it into code ( https://playwright.dev/ ).
[ Parent | on HN ]