News for hackers. Stuff that matters.

A comment on Amazon's AI crawler is making my Git server unstable. Parent comment

Re:Amazon's AI crawler is making my Git server unstable by jsheard on Saturday January 18, 2025 @02:24PM (#42750711)
Amazon does publish every IP address range used by AWS, so there is the nuclear option of blocking them all pre-emptively.

https://docs.aws.amazon.com/vpc/latest/userguide/aws-ip-rang...

[ Parent | on HN ]
Re:Amazon's AI crawler is making my Git server unstable by xena on Saturday January 18, 2025 @02:27PM (#42750731)
I'd do that, but my DNS is via route 53. Blocking AWS would block my ability to manage DNS automatically as well as certificate issuance via DNS-01.
[ Parent | on HN ]
Re:Amazon's AI crawler is making my Git server unstable by actuallyalys on Saturday January 18, 2025 @02:38PM (#42750793)
They list a service for each address, so maybe you could block all the non-Route 53 IP addresses. Although that assumes they aren’t using the Route 53 IPs or unlisted IPs for scraping (the page warns it’s not a comprehensive list).

Regardless, it sucks that you have to deal with this. The fact that you’re a customer makes it all the more absurd.

[ Parent | on HN ]
Re:Amazon's AI crawler is making my Git server unstable by unsnap_biceps on Saturday January 18, 2025 @02:34PM (#42750771)
If you only block new inbound requests, it shouldn't impact your route 53 or DNS-01 usage.
[ Parent | on HN ]