Was rate limiting not a concern? How often did you run it? I'm sure you had to keep some state too?
Their website was pretty trivial, but I beefed up the script to check for failure status code and also notify me if it wasn't 200.
I ran it every minute.
I had to include a unique uuid to cache bust, but other than that there wasn't any state.
I wrote about it here https://plainice.com/webscrape-notifications
However, the jobs were first come, first serve so he was waking up at 4 am and constantly refreshing for hours trying to be the first one.
When I heard about it, I quickly whipped up a `curl | grep && send_notif` (used pushback.io for notifs) and it helped him not have to worry so much.
When a new job posting finally came along he was the first in line and got the job :)