Show HN: Open-source indexing of Discord questions into Google

Re:Show HN: Open-source indexing of Discord questions into G... by mdaniel on Tuesday March 28, 2023 @12:58PM (#35344189)

related to that, I somehow thought the sitemap protocol was that any such thing was referenced from /robots.txt (https://www.sitemaps.org/protocol.html#informing) but your site's robots.txt is 404 (https://www.answeroverflow.com/robots.txt) even though https://www.answeroverflow.com/sitemap.xml exists. One can certainly question whether "weekly" is the correct changefreq for a discord server url but maybe that's just a concession to some rate limit?

[ Parent | on HN ]

Re:Show HN: Open-source indexing of Discord questions into G... by rhyssullivan1 on Tuesday March 28, 2023 @02:22PM (#35345568)

Thanks for pointing that out! The robots.txt file was in the location, that got messed up a bit in the move over the open source version but is fixed now

For the weekly changefreq on Discord server url, are you arguing that's too frequent or too infrequent? On the server page, it has recently indexed questions so part of the reasoning behind that being weekly is to capture those recently indexed questions https://www.answeroverflow.com/c/143867839282020352

Archived threads get a change freq of monthly since they don't change that often and active threads have a changfreq of daily as there is lots of activity going on in them

[ Parent | on HN ]