Re:LastPass edits robots.txt to make support articles diffic...
by tagersenim on Tuesday February 28, 2023 @06:15AM
(#34968081)
From the article:
All of today's support bulletins are not easy to find, with none of them listed in search engines, as the company added <meta name="robots" content="noindex"> HTML tags to the document to prevent them from being indexed by search engines.
Re:LastPass edits robots.txt to make support articles diffic...
by mtmail on Tuesday February 28, 2023 @06:57AM
(#34968332)
The submission title includes "LastPass edits robots.txt" but https://support.lastpass.com/robots.txt doesn't seem to exclude those URLs and the article doesn't mention robots.txt
2) To clarify a bit, afaik, robots.txt is a request / suggestion, a nice to have if you will. It doesn't prevent indexing.