Re:Almost all searches on my independent search engine are n...
by kmeisthax on Monday May 16, 2022 @10:58AM
(#31398372)
...so you're going to write your own HTTP requests? Encrypt your traffic and validate certificates by hand? Toggle in each TCP header from a memory debugger?
Most of the Internet is bots because humans don't actually generate HTTP traffic - they fire up a bot called a "browser" to do it for them. The challenge for anti-spam is to distinguish which bots are currently being directly controlled by humans and which ones are not-so-directly controlled by such. This isn't even a hard line; I've frequently hit Hacker News' bot detection just by upvoting a comment and then clicking reply too quickly.
Most of the Internet is bots because humans don't actually generate HTTP traffic - they fire up a bot called a "browser" to do it for them. The challenge for anti-spam is to distinguish which bots are currently being directly controlled by humans and which ones are not-so-directly controlled by such. This isn't even a hard line; I've frequently hit Hacker News' bot detection just by upvoting a comment and then clicking reply too quickly.