News for hackers. Stuff that matters.
Re:How to scrape anything on the web and not get caught by jstarfish on Sunday April 22, 2018 @04:35AM (#16895286)
IPs are naive and too broad. Force users to authenticate to see your content. Unauthenticated viewers get to see only a scrambled version. Now you can track usage by username.

Establish a baseline as to what constitutes "normal" browsing.

Any user exceeding a threshold of page requests gets rate limited/banned.

Stopping people from recording public information is unrealistic, but you can certainly make it more of a pain in the ass.

There are no comments on this comment yet.