News for hackers. Stuff that matters.

A comment on A special build of curl that can impersonate Chrome and Firefox. Parent comment

Re:A special build of curl that can impersonate Chrome and F... by NavinF on Thursday April 03, 2025 @06:38PM (#43576317)
Last time I saw someone complaining about scrapers, they were talking about 100gib/month. That's 300kbps. Less than $1/month in IP transit and ~$0 in compute. Personally I've never noticed bots show up on a resource graph. As long as you don't block them, they won't bother using more than a few IPs and they'll backoff when they're throttled
[ Parent | on HN ]
Re:A special build of curl that can impersonate Chrome and F... by marcus0x62 on Thursday April 03, 2025 @06:55PM (#43576459)
For some sites, things are a lot worse. See, for example, Jonathan Corbet's report[0].

0 - https://social.kernel.org/notice/AqJkUigsjad3gQc664

[ Parent | on HN ]
Re:A special build of curl that can impersonate Chrome and F... by NavinF on Saturday April 05, 2025 @04:22PM (#43596528)
He provides no info. req/s? 95%ile mbps? How does he know the requests come from an "AI-scraper" as opposed to a normal L7 DDoS? LWN is a pretty simple site, it should be easy to saturate 10G ports
[ Parent | on HN ]
Re:A special build of curl that can impersonate Chrome and F... by lmz on Thursday April 03, 2025 @10:23PM (#43577683)
How can you say it's $0 in compute without knowing if the data returned required any computation?
[ Parent | on HN ]
Re:A special build of curl that can impersonate Chrome and F... by NavinF on Saturday April 05, 2025 @04:25PM (#43596554)
Look at the sibling replies. All the kvetching comes from blogs and simple websites, not the ones that consume compute per request
[ Parent | on HN ]
Re:A special build of curl that can impersonate Chrome and F... by nonrandomstring on Friday April 04, 2025 @03:39AM (#43579340)
Didn't rachelbytheebay post recently that her blog was being swamped? I've heard that from a few self-hosting bloggers now. And Wikipedia has recently said more than half of traffic is noe bots. ARe you claiming this isn't a real problem?
[ Parent | on HN ]
Re:A special build of curl that can impersonate Chrome and F... by NavinF on Saturday April 05, 2025 @04:24PM (#43596544)
How exactly can a blog get swamped? It takes ~0 compute per request. Yes I'm claiming this is a fake problem
[ Parent | on HN ]