Using regex in python to scrape some data from a website works just fine for me... shrugs

It’s an acceptable choice for scraping, because with scraping you usually go into it with the expectation that it will break and need updates sometimes as the scraped site changes. The warning bells are more for things like validating user input on your own site, where there might be security implications of an impossible parsing task.

And I use a handgun to hunt houseflies in my motel room after a week long crystal meth binge. Works just fine for me.

Is there ever a good reason to use regex to do a web scraper instead of using a proper prebuilt parser and walking the DOM?

regex is your hammer and you just by chance happen to have a really nail shaped task to do?

Performance, perhaps?

beautiful soup is a better tool.

Regex is fine for grabbing something in a page that you have looked at yourself.

Parsing millions of pages you don't have this option, you need something robust, a tool that is flexible, that doesn't barf out too many errors, that is quick.

BS4, requests and dataset are my go-to tools for simple scraping. Back when Bethesda pulled the paid mod nonsense on the Steam store I wrote a simple scraper to track and digest the changes people were making to their Skyrim reviews so I could tease out who was most offended, core fans with a lot of hours logged on the game or people who were just piling on the outrage bandwagon. I should dig back into that data some time.

Anyway, got the whole thing done in about 200 lines. Not bad to scrape 50,000 pages. Fun times.

yep me too, no problem.