News for hackers. Stuff that matters.
Re:Show HN: Reveddit.com: Improving online discourse with tr... by dmix on Saturday November 05, 2022 @03:08AM (#33478753)
One more question I just thought of. When I use services like Reveddit I often notice [removed too quickly] where I'm guessing moderators get to it before the scraper can archive it.

What type of timeframe do you guys typically work with before it gets removed? Any ideas on how to improve the capture times?

Also are you looking for dev help with your project?

Re:Show HN: Reveddit.com: Improving online discourse with tr... by rhaksw on Saturday November 05, 2022 @03:32AM (#33478871)
> When I use services like Reveddit I often notice [removed too quickly] where I'm guessing moderators get to it before the scraper can archive it. What type of timeframe do you guys typically work with before it gets removed? Any ideas on how to improve the capture times?

Current capture time is noted in the status on Reveddit.com/info [1]. That blue box will show up in threads when the capture time is delayed more than the standard, which I think is typically under a minute.

The content for [removed] comments in threads comes from another third party service called Pushshift. It isn't able to get many auto-removed comments (such as those removed by Automoderator), and that service sometimes goes down, may fail to ingest something, etc.

In an attempt to fill some of these gaps, Reveddit can restore some unarchived comments by searching the user pages of other commenters in the thread. And if you visit Reveddit via a direct link to a removed comment, such as from the button that is provided under such comments by the Reveddit Linker extension [2], then it will try to auto-load those comments this way. Otherwise you can click Restore to do it manually.

Almost all of the features on the site should be documented in either a bubble or the FAQ. If you see something that you think needs a help bubble, or a better description, let me know.

> Also are you looking for dev help with your project?

Sure! The code is on Github. Feel free to brainstorm via issues, email, or submit PRs. My main focus now is raising awareness so it would be great to have backfill on keeping the code functional.

[1] https://www.reveddit.com/info

[2] https://www.reveddit.com/add-ons/linker

Re:Show HN: Reveddit.com: Improving online discourse with tr... by dmix on Saturday November 05, 2022 @03:37AM (#33478898)
> Reveddit can restore some unarchived comments by searching the user pages of other commenters in the thread.

Clever

Re:Show HN: Reveddit.com: Improving online discourse with tr... by rhaksw on Saturday November 05, 2022 @03:47AM (#33478954)
There are a lot of fun things you find when you start working on this. Virtually nobody else is doing it because they all think it's wrong without realizing it's the right thing to do.

For example, you could reverse engineer automod configurations by scraping user pages with a headless browser, storing all of the auto-removed comments, and then looking for the most common n-grams in a given subreddit. Alternatively you could make the code work "server-side" so some sort of json is returnable, though that would take more work.

And I haven't even mentioned the history pages, which may indicate where users and moderators have disagreed the most:

https://www.reveddit.com/v/worldnews/history/

Plus, this is only one social media site. Every site does this, and there's probably a way to reveal the shenanigans going on in a lot more places.