News for hackers. Stuff that matters.
Re:XPath Scraping with FreshRSS by Nextgrid on Tuesday December 26, 2023 @08:57PM (#38778244)
I wonder how this would work with more and more sites behind Cloudflare and stuff. Websites really don't want this since in today's economy, wasting a human's time is paramount (they call it "engagement"). A computer, even working on behalf of a human is not enough.
Re:XPath Scraping with FreshRSS by 082349872349872 on Tuesday December 26, 2023 @09:33PM (#38778467)
Fortunately I find the value of a website to be often in inverse relation to how much it cares about engagement.

(fwiw everyone I've wanted to follow has already had a feed; it's just that sometimes I've had to grovel around with View Source to find it)

Re:XPath Scraping with FreshRSS by derac on Tuesday December 26, 2023 @10:30PM (#38778794)
Curl impersonate gets right through Cloudflare. ;)