News for hackers. Stuff that matters.
Re:ByteDance’s Bytespider is scraping at much higher rates t... by diggan on Monday October 07, 2024 @07:11AM (#41764846)
> Does any of these scrapers uniquely and unambiguously identify themselves as a bot?

It seems like all of them do, yeah: https://github.com/eob/isai/blob/b9060db7dc1a7789b322b8c2838...

Not sure if they're really "scrapers" though, if they're initiated by a user for a single webpage/website, more like "user-agents" in that case, unless it automatically fans out from there to get more content.

There are no comments on this comment yet.