Show HN: A free, privacy preserving, archive of public Discord servers

Re:Show HN: A free, privacy preserving, archive of public Di... by treyd on Tuesday May 20, 2025 @01:45AM (#44038161)

Would you consider making regular dumps of the database available in sharded torrents like Anna's Archive does so that users can back up the data themselves for preservation purposes? This would complicate retroactively removing users' activity, but that data could already be scraped.

And related, I'd like to be able to run this locally for exports of guilds that I'm on myself. Is that even possible with the architect you've built?

[ Parent | on HN ]

Re:Show HN: A free, privacy preserving, archive of public Di... by searchcord on Tuesday May 20, 2025 @01:49AM (#44038176)

Hey,

This is absolutely something I want to do, but at the guild level. The database itself is over 13TB which is much to large to create regular exports of. I will probably provide a SQLite export of each guild, regenerated each week/month. Anyone is free to download whatever they want in real time from the API.

Thanks for your question!

[ Parent | on HN ]

Re:Show HN: A free, privacy preserving, archive of public Di... by squigz on Tuesday May 20, 2025 @04:13AM (#44039054)

Big +1 for dumps

You might try reaching out to Anna's Archive and see if this would be a dataset they'd be interested in helping host/distribute. I think they'd agree that such data is important and should be archived.

[ Parent | on HN ]

Re:Show HN: A free, privacy preserving, archive of public Di... by pilimi_anna on Monday May 26, 2025 @03:20PM (#44100681)

Yes, we'd happily host and torrent this.

[ Parent | on HN ]

Re:Show HN: A free, privacy preserving, archive of public Di... by treyd on Tuesday May 20, 2025 @10:36AM (#44042137)

I was imagining something like a month-by-month historical export so only recent data would need to be regularly exported.

SQLite dumps would be awesome to work with though.

[ Parent | on HN ]