Re:26% of the top websites are now blocking GPTBot
by rolph on Wednesday September 27, 2023 @01:33PM
(#37678132)
[delayed]
Re:26% of the top websites are now blocking GPTBot
by akira2501 on Wednesday September 27, 2023 @01:59PM
(#37678533)
You can also just do that for no particularly good reason. Language models aren't going to go anywhere particularly worthwhile if this is their primary mode of data gathering.
Re:26% of the top websites are now blocking GPTBot
by hinkley on Friday September 29, 2023 @12:56AM
(#37699438)
I needed to test whether an app could decompress inputs over 2GB properly. I was able to create a zip file that was about 45k to do it. I think the record is something like 5k, but 50k fits just fine in a repository.
Set up direct Gzip sendfile, and that’ll keep them busy for a while.
If I’m paying for servers by the hour and half of them are run just to keep the bots happy, I’m going to have opinions about who is allowed to visit my pages and for what reason. Mooches don’t have a moral leg to stand on.
There is a danger though if certain types of sites are more likely to block GPTBot than others, because that would end up skewing the data set that it trains off, which could have longer term impacts on all the content generated with it. If all the good quality sites block it and the sites full of AI generated junk don't, then that sounds like a downward spiral.