News for hackers. Stuff that matters.
Re:Perplexity is using stealth, undeclared crawlers to evade... by bee_rider on Monday August 04, 2025 @02:50PM (#44789930)
> Cloudflare banning bad actors has at least made scraping more expensive, and changes the economics of it - more sophisticated deception is necessarily more expensive. If the cost is high enough to force entry, scrapers might be willing to pay for access.

I think this might actually point at the end state. Scraping bots will eventually get good enough to emulate a person well enough to be indistinguishable (are we there yet?). Then, content creators will have to price their content appropriately. Have a Patreon, for example, where articles are priced at the price where the creator is fine with having people take that content and add it to the model. This is essentially similar to studios pricing their content appropriately… for Netflix to buy it and broadcast it to many streaming users.

Then they will have the problem of making sure their business model is resistant to non-paying users. Netflix can’t stop me from pointing a camcorder at my TV while playing their movies, and distributing it out like that. But, somehow, that fact isn’t catastrophic to their business model for whatever reason, I guess.

Cloudflare can try to ban bad actors. I’m not sure if it is cloudflare, but as someone who usually browses without JavaScript enables I often bump into “maybe you are a bot” walls. I recognize that I’m weird for not running JavaScript, but eventually their filters will have the problem where the net that captures bots also captures normal people.

Re:Perplexity is using stealth, undeclared crawlers to evade... by Barbing on Tuesday August 05, 2025 @02:11AM (#44794815)
>Then they will have the problem of making sure their business model is resistant to non-paying users. Netflix can’t stop me from pointing a camcorder at my TV while playing their movies, and distributing it out like that. But, somehow, that fact isn’t catastrophic to their business model for whatever reason, I guess.

Interested to see some LLM-adverserial equivalent of MPAA dots![1]

[1] https://en.wikipedia.org/wiki/Coded_anti-piracy

Re:Perplexity is using stealth, undeclared crawlers to evade... by skeezyboy on Tuesday August 05, 2025 @05:57AM (#44796159)
Netflix CAN "stop you from pointing a camera at your TV and distributing it" because of copyright law.
Re:Perplexity is using stealth, undeclared crawlers to evade... by account42 on Tuesday August 05, 2025 @09:25AM (#44797677)
Which is also how AI scrapers should be solved. Papering over the issue with technological "solutions" only hurts real users.
Re:Perplexity is using stealth, undeclared crawlers to evade... by skeezyboy on Tuesday August 05, 2025 @11:59AM (#44799751)
in the UK at least, that has been recognized as fair use.