News for hackers. Stuff that matters.
Re:Ask HN: Why Is My Scraping So Slow? by psnehanshu on Wednesday May 10, 2023 @01:58AM (#35884132)
Idk how cloudflare rate limiting can impact this unless the webflow site is behind cloudflare? Can it be removed?

It may also be that webflow rate limits bot traffic? Try spoofing the user agent with a popular browser's[1].

But why scrape? Webflow allows to export the code[2]. But it may still require premium subscription, I haven't looked thoroughly.

[1] https://techblog.willshouse.com/2012/01/03/most-common-user-...

[2] https://university.webflow.com/lesson/code-export

There are no comments on this comment yet.