News for hackers. Stuff that matters.
Re:Wikipedia's robots.txt by ufo on Tuesday July 16, 2019 @07:12PM (#20454782)
> Please do not remove the space at the start of this line, it breaks the rendering.

Does anyone know what this comment means? It is towards the end of the file.

Re:Wikipedia's robots.txt by bawolff on Tuesday July 16, 2019 @08:00PM (#20455114)
Its because the robots.txt file is created by combining a static file ( https://github.com/wikimedia/operations-mediawiki-config/blo... ) with an on-wiki page so that different language wikipedias can manage the file themselves - https://en.wikipedia.org/wiki/MediaWiki:Robots.txt

In wikisyntax, starting a line with a space ensure's that its in a pre tag, so this was a way to make the onwiki page show the entire thing in a pre tag and not use normal formatting. This seems to be broken by actually starting a literal pre tag inside the first line, but I'm pretty sure this used to work as a way to display the entire thing in a pre tag.

For the curious, the code generating the robots.txt file is at https://github.com/wikimedia/operations-mediawiki-config/blo...