Re:New headless Chrome has been released and has a near-perf...
by inetknght on Sunday February 19, 2023 @11:25AM
(#34858936)
> What do I do when someone hits my content-rich Wordpress blog with a scraper that hits 100 pages a second to download my content, and my database falls over
It's a blog. Blogs are not complex. Why is your blog's database so awfully designed that 100 pages a second causes it to fall over?
> leading to real, legitimate users being unable to use my site?
You assume that a scraper is not a legitimate user. I argue otherwise. If you don't want a scraper to use your use then put your site behind a paywall.
> What if it’s not a legitimate scraper but someone with hundreds of proxies uses them to DDOS my site for days?
If it's a network bandwidth problem, then a reverse proxy (eg, CDN) solves that.
> Should I sacrifice my uptime to protect the freedom of those unwilling to attest that they’re running on real hardware?
All software runs on real hardware. What is your exact question?
I am accessing this site in a virtual machine. I could be doing it with a headless browser. Why does that matter at all?
It's a blog. Blogs are not complex. Why is your blog's database so awfully designed that 100 pages a second causes it to fall over?
> leading to real, legitimate users being unable to use my site?
You assume that a scraper is not a legitimate user. I argue otherwise. If you don't want a scraper to use your use then put your site behind a paywall.
> What if it’s not a legitimate scraper but someone with hundreds of proxies uses them to DDOS my site for days?
If it's a network bandwidth problem, then a reverse proxy (eg, CDN) solves that.
> Should I sacrifice my uptime to protect the freedom of those unwilling to attest that they’re running on real hardware?
All software runs on real hardware. What is your exact question?
I am accessing this site in a virtual machine. I could be doing it with a headless browser. Why does that matter at all?