News for hackers. Stuff that matters.
Re:ByteDance’s Bytespider is scraping at much higher rates t... by Ironlikebike on Monday October 07, 2024 @09:15AM (#41765678)
In my last job, we observed ByteDance scraping TBs of OS testing data using the restful API that our OSS community front-end was using to serve it's CI result to the OSS community. The scraping was so relentless it was causing performance problems. We were also worried they were going to cause us large network egress fees as well. We specifically locked down the API after that, and anyone who wanted to use results had to ask explicit permission and be granted access.

There are no comments on this comment yet.