News for hackers. Stuff that matters.

sfsdfsdfasfasd

Re:The Evolution of Marginalia's Crawling on Tuesday August 23, 2022 @11:46AM (#32566643)
It's very simple to write a basic ActivityPub crawler (I have done this), but you'd go from outbox to outbox, not inbox to inbox. Regardless, it's all just JSON, so it is more straightforward to crawl than the web.

I tried writing a crawler to map the Fediverse (to discover homeservers) but I discovered quickly that most of the content in the Fediverse is extremist content, both left-wing and right-wing -- it's basically all tankies and Nazis, with a few techies in a bubble using mastodon.social, who think Fediblock is a solution to this problem.

Oh right, and there are a ton of pedo instances too, mostly based in Japan where "loli" is legal. Due to the way ActivityPub publishes federated content, I didn't feel comfortable running a homeserver, because a pedo instance could federate illegal content to my homeserver and get me arrested and charged for possession of content I did not request and do not want to store on my computers, and if I'm not monitoring what content is getting federated to my timeline, I might not even know it's there. Too risky.

Demoralized, I abandoned the project, and I don't think much about ActivityPub anymore.