mintzworld

Oh, the irony of using SeleniumBase to make Playwright stealthy!

Even a single script that performs actions too quickly on a website can trigger anti-bot measures, even if the bot isn't detected directly.

Why pay to solve CAPTCHAs when SeleniumBase can bypass them for free? SeleniumBase can also "solve" CAPTCHAs (such as Cloudflare via click).

On the topic of scraping grocery sites, here's an example of bypassing bot-detection on Albertsons: https://github.com/seleniumbase/SeleniumBase/blob/master/exa... (A demo of that is in https://www.youtube.com/watch?v=Mr90iQmNsKM)

PyAutoGUI is the optimal tool for clicking things inside of closed shadow-root elements, which are hidden to JavaScript. Can use CDP for clicking other elements.

The "CDP Mode" used for bypassing CAPTCHAs and bot-detection has it's own ReadMe within SeleniumBase: https://github.com/seleniumbase/SeleniumBase/blob/master/exa... And there's a recent YouTube video with live demos: https://www.youtube.com/watch?v=Mr90iQmNsKM

According to live demos seen in https://www.youtube.com/watch?v=Mr90iQmNsKM, it'll bypass Cloudflare, Akamai, Shape Security, DataDome, Incapsula, Kasada, and PerimeterX.

SeleniumBase is free, open-source, can bypass CAPTCHAs with a few lines of code, and it works from the free tier of GitHub Actions.

"Service workers can only be used over HTTPS, or on localhost for development" - That's a known Gatsby issue: https://github.com/gatsbyjs/gatsby/issues/10369

It's possible to be testing a live version of your own site when you can't easily change the Same-Origin policy for just yourself.

Some websites have a Same-Origin policy that won't allow rendering in an iframe. The CORS proxy lets you bypass the restriction.

There are a few websites that will detect unusual behavior from browsers living inside of browsers. Although it does seem to work for most websites.

Selenium to the rescue!

SeleniumBase (which wraps Selenium) adds reliability by improving existing methods so that the browser waits for page elements to fully load before interacting with them. It might be worth trying some of the examples from the SeleniumBase GitHub page to see if you still feel that way.

SeleniumBase methods wait for page elements to fully load before interacting with them, which brings test reliability. There's also built-in mobile testing and cross-browser testing features included: https://github.com/seleniumbase/SeleniumBase/blob/master/hel...