Re:Wikimedia is moving to Gitlab
by dragonwriter on Wednesday October 28, 2020 @01:07PM
(#24921295)
> Because spammers fill the publicly viewable things-- like snippets and user profiles-- with spam. If it can be viewed without logging in, then they get it indexed with Google and it dilutes the search results.
Isn't then the problem not that it is viewable but that it's not excluded from indexing by robots.txt?
Isn't then the problem not that it is viewable but that it's not excluded from indexing by robots.txt?